REVIEW 4 major objections 4 minor 51 references
Permutation-Free High-Order Interaction Tests
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that kernel high-order interaction tests can be made permutation-free through V-statistics and cross-centring, with null statistics that follow a standard normal distribution.
desk verdict The permutation-free construction and the empirical work are genuinely useful, but the central N(0,1) claim for d>2 is asserted rather than proved, so the paper needs major theoretical revisions before acceptance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is cross-centring combined with V-statistics under data splitting. For 2n i.i.d. samples, the kernel matrix K is centred only across the two halves as K = C_u K C_l, where C_u and C_l subtract the mean of the upper and lower half respectively; this makes the cross-centred kernel asymptotically equivalent to the usual centred kernel while leaving the statistic with a non-degenerate limiting variance. The test statistics are quadratic forms 1^T (Hadamard product of cross-centred kernel blocks) 1 over the cross half-block, normalised by the empirical standard deviation of the per-row contributions. This self-normalised ratio is what converts the degenerate null of a V-statistic into a standard normal pivot, so a single computation replaces the permutation distribution.
What would settle it
Generate i.i.d. data under a d=4 joint-independence null, compute xdHSIC over 10,000 replications at n=200 and n=500, and compare the empirical distribution with N(0,1) using a Q-Q plot and a Kolmogorov-Smirnov test; if the tails are heavier or the variance is not 1, the consistency claim for $s_I^{2}$ is falsified. The analogous check can be run for xLI and xSI under a null subhypothesis such as P1234 = P12 P34.
Extended reading notes
Core claim
The central claim is that, for any d, joint independence and partial-factorisation hypotheses can be tested by V-statistics built from Hadamard products of cross-centred kernel matrices on two disjoint halves of the sample. With this sample split, each statistic takes the form $\sqrt$(n) times an unnormalised cross-block sum divided by an empirical standard deviation, and the paper claims this ratio converges to N(0,1) under the corresponding null: joint independence for xdHSIC, each singleton-factorisation subhypothesis for xLI, and each two-block partition for xSI. xSI vanishes if and only if the joint distribution factorises into any two blocks, and combining xLI with xSI covers all 2^(d-1)-1 bipartitions needed for the complete factorisation test. The same construction specialised to d=2 yields a simpler pairwise independence statistic that the paper reports to have higher power than the existing permutation-free xHSIC.
Load-bearing premise
The treatment assumes, without proof, that the empirical variance estimators $s_I^{2}$, $s_L^{2}$, and $s_S^{2}$ consistently estimate the asymptotic variance of the cross-block V-statistics for d>2, so that the normalised statistics actually converge to N(0,1).
Editorial extensions
If this is right
- xdHSIC, xLI, and xSI each reject their subhypotheses by comparing a single statistic with N(0,1), so p-values become deterministic and the computation is p-fold faster than using p permutations.
- All three tests run in O(d n^2), and the xLI+xSI strategy keeps complexity quadratic in n regardless of d, whereas the permutation-based Streitberg test scales combinatorially with d.
- For d=2, the V-statistic and cross-centring formulation gives a pairwise independence statistic that is simpler and reported to be more powerful than the earlier permutation-free xHSIC.
- In score-based causal discovery, xdHSIC recovers the correct DAG with fewer samples than permutation-based dHSIC, and in the XOR-based feature selection example only xSI+xLI selects the ground-truth feature set.
- On S&P 500 daily returns, the method screens 2- to 5-way interactions within and across sectors, finding systematically higher within-sector high-order interactions, especially for Utilities and Energy.
Reading between the lines
- An extension the paper leaves implicit is that the fixed N(0,1) null makes whole-family multiple-testing control over the exponential number of partitions straightforward: all subtests share one critical value, so composite rejection can be computed without generating a permutation null for each partition.
- The data-splitting construction suggests a direct route to stationary time series and network data by replacing the two independent halves with two blocks separated by a gap; the paper notes permutation-based versions for such data exist but leaves the permutation-free version open.
- Because the null pivot is a standard normal, formal power analysis could be derived analytically from the mean shift of the cross-block sum under contiguous alternatives; the paper does not provide such a power formula.
- Combining cross-centring with low-rank kernel approximations such as random Fourier features or the Nystroem method should reduce the O(d n^2) cost to roughly O(d n m), which would make interaction screening feasible on very large feature sets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three permutation-free kernel tests for high-order interactions: xdHSIC for joint independence, xLI for Lancaster factorisations, and xSI for Streitberg/complete factorisations. The statistics are formed by splitting the sample, cross-centring kernel matrices, and using V-statistic-type bilinear forms; the authors claim that each normalised statistic has a standard normal null distribution, so a single computation replaces p permutations. The paper reports O(dn^2) complexity, presents synthetic and real-data experiments (causal discovery, feature selection, stock returns), and provides code.
Significance. If the normality claim were established, the paper would make a useful contribution: it would extend permutation-free testing beyond pairwise independence to d>2, make high-order interaction testing practical in combinatorial searches, and the d=2 V-statistic reformulation xHSICV is a clean simplification. The empirical evaluation is broad, and public code is a strength. However, the central theoretical claim is not proved for d>2, and the variance estimators are incompletely specified; until that is fixed, the methodological contribution is not established.
major comments (4)
- [Section 3.1, Definition 3.1 and Hypothesis 3.2] The central claim that xdHSIC is asymptotically standard normal under H0 for d>2 is asserted, not proved. The citation to Shekhar et al. (2023) after Hypothesis 3.2 covers the pairwise cross U-statistic, whereas xdHSIC is a d-dimensional cross-block V-statistic with a different variance estimator s_I^2. No theorem states the limiting distribution for d>2, and Appendix A.1 only verifies the O(dn^2) complexity; it does not establish consistency of s_I^2 or asymptotic normality. Since the permutation-free property is exactly this null distribution, the missing derivation is load-bearing.
- [Definitions 3.7 and 3.10] The displayed definitions of s_L^2 and s_S^2 contain unreadable placeholder tokens rather than complete expressions, so the normalised statistics xLI and xSI cannot be checked from the manuscript. No consistency proof for either variance estimator is provided; the remarks in Sections 3.2.1 and 3.2.2 only assert the N(0,1) null. This must be repaired and accompanied by a proof.
- [Lemma 3.5 and Section 3.2] Lemma 3.5 establishes asymptotic equivalence of a single cross-centred kernel to the original kernel. The Lancaster and Streitberg statistics, however, are built from Hadamard products of separately cross-centred kernels, e.g. \overline{K^m} \odot \bigodot_{i\ne m}\overline{K^i} in Definition 3.7. Equivalence of the individual factors does not imply equivalence of their Hadamard product, and no argument reduces the null distribution of such products to the pairwise xHSIC framework. Appendix A.3 and A.4 merely assert that the products can be viewed as single kernels; this addresses complexity, not distributional convergence.
- [Propositions 3.8, 3.11, and Table 1] These statements conflict. Proposition 3.8 claims xLI for all H_pi_L is O(dn^2), while Table 1 reports O(d^2 n^2); Proposition 3.11 claims xSI is O(dn^2), while Table 1 reports O(2^d d n^2) for xSI+xLI. Since xLI runs d singleton subtests and xSI covers the non-singleton bipartitions, the all-subtests complexities in the table are the relevant ones, and the propositions should be corrected to state per-subtest complexity.
minor comments (4)
- [Definitions 3.1, 3.7, 3.10] The normalised statistics are written with a denominator s_I, s_L, or s_S even though the displayed objects are s_I^2, s_L^2, s_S^2; please clarify that the denominator is the square root of the displayed variance estimator.
- [Appendix B] The displayed implications use corrupted arrow tokens (e.g., '/Leftr⫯g⊸tl⫯ne⇒') and need to be typeset correctly.
- [Section 5.1] The sentence 'xSI+xLI can be computed in less that 1 second' contains a typo; it should read 'less than'.
- [Appendix C] The claim that xHSICV has 'higher power' than xHSIC is based on a single simulation (Fig. 9); please qualify this as an empirical observation for the considered setting.
Circularity Check
No significant circularity: the tests do not fit parameters to their own predictions, and the heavy reliance on prior work is external or mathematical, not a re-labeling of the target result.
full rationale
The paper's central claim is that xdHSIC, xLI, and xSI converge to a standard normal under the respective null hypotheses, enabling a permutation-free test. This claim is not obtained by fitting a parameter to a dataset and then predicting a closely related quantity; the statistics are direct functions of the data and the kernels. The construction is anchored to an external result: 'The construction of xdHSIC ensures it follows N(0, 1) under the null (Shekhar et al., 2023).' The authors' own prior work (Liu et al., 2023b) is cited for the Lancaster and Streitberg vanishing properties and for the partition-lattice facts that justify the composite hypotheses. These are published mathematical results, not fitted values, and they do not by themselves assert the d>2 normality of the new statistics, so they are not used as a circular substitute for the derivation. The main weakness is that the d>2 normality is asserted rather than proved: the text says 'all with a fixed null distribution (standardised to N(0, 1))' in the Remark after Definition 3.7, and Appendix A proves only the equivalence of the cross-centred kernel and the O(dn^2) complexity, not the distributional convergence for the Hadamard-product statistics. This is a missing proof, which is a correctness and rigor concern, not a circularity: the paper does not define the statistic in terms of the claimed normal limit, nor does it rename a fitted input as a prediction. The garbled placeholders in the displayed formulas for s_L^2 and s_S^2 are manuscript defects and do not constitute a circular step. Overall, no load-bearing step reduces by construction to its own inputs, so the paper is not circular beyond a minor reliance on self-citation for background mathematical facts.
Assumptions & free parameters
assumptions (4)
- domain assumption Samples are i.i.d.
- standard math Kernels are characteristic so that kernel mean embeddings are injective.
- ad hoc to paper Cross-centred kernel is asymptotically equivalent to the original kernel (Lemma 3.5).
- domain assumption Rejecting a composite factorisation null if all subtests reject at level alpha is valid and less conservative than Bonferroni.
Cite this review
Pith. "Pith review of Permutation-Free High-Order Interaction Tests." pith.science (2026). https://pith.science/paper/HNDBM6AX
@misc{pith2026250605963,
author = {Pith},
title = {Pith review of: Permutation-Free High-Order Interaction Tests},
year = {2026},
howpublished = {\url{https://pith.science/paper/HNDBM6AX}},
note = {Machine review of arXiv:2506.05963}
}
abstract
Kernel-based hypothesis tests offer a flexible, non-parametric tool to detect high-order interactions in multivariate data, beyond pairwise relationships. Yet the scalability of such tests is limited by the computationally demanding permutation schemes used to generate null approximations. Here we introduce a family of permutation-free high-order tests for joint independence and partial factorisations of $d$ variables. Our tests eliminate the need for permutation-based approximations by leveraging V-statistics and a novel cross-centring technique to yield test statistics with a standard normal limiting distribution under the null. We present implementations of the tests and showcase their efficacy and scalability through synthetic datasets. We also show applications inspired by causal discovery and feature selection, which highlight both the importance of high-order interactions in data and the need for efficient computational methods.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Ali, M. M. and Giaccotto, C. The identical distribution hypothesis for stock market prices—location-and scale-shift alternatives. Journal of the American Statistical Association, 77 0 (377): 0 19--28, 1982
work page 1982
-
[2]
Arnaudon, A., Peach, R. L., Petri, G., and Expert, P. Connecting hodge and sakaguchi-kuramoto through a mathematical framework for coupled oscillators on simplicial complexes. Communications Physics, 5 0 (1): 0 211, 2022
work page 2022
-
[3]
Networks beyond pairwise interactions: structure and dynamics
Battiston, F., Cencetti, G., Iacopini, I., Latora, V., Lucas, M., Patania, A., Young, J.-G., and Petri, G. Networks beyond pairwise interactions: structure and dynamics. Physics Reports, 874: 0 1--92, 2020
work page 2020
-
[4]
The physics of higher-order interactions in complex systems
Battiston, F., Amico, E., Barrat, A., Bianconi, G., Ferraz de Arruda, G., Franceschiello, B., Iacopini, I., K \'e fi, S., Latora, V., Moreno, Y., et al. The physics of higher-order interactions in complex systems. Nature Physics, 17 0 (10): 0 1093--1098, 2021
work page 2021
-
[5]
Benson, A. R., Gleich, D. F., and Leskovec, J. Higher-order organization of complex networks. Science, 353 0 (6295): 0 163--166, 2016
work page 2016
-
[6]
Bick, C., Gross, E., Harrington, H. A., and Schaub, M. T. What are higher-order networks? SIAM Review, 65 0 (3): 0 686--731, 2023
work page 2023
-
[7]
Bi \'n kowski, M., Sutherland, D. J., Arbel, M., and Gretton, A. Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018
arXiv 2018
-
[8]
B \"o ttcher, B. Dependence and dependence structures: estimation and visualization using the unifying concept of distance multivariance. Open Statistics, 1 0 (1): 0 1--48, 2020
work page 2020
Show all 51 references
-
[9]
and Zhang, X
Chakraborty, S. and Zhang, X. Distance metrics for measuring joint dependence with application to causal inference. Journal of the American Statistical Association, 2019
2019
-
[10]
and Gretton, A
Chwialkowski, K. and Gretton, A. A kernel independence test for random processes. In International Conference on Machine Learning, pp.\ 1422--1430. PMLR, 2014
2014
-
[11]
Unifying feature-based explanations with functional anova and cooperative game theory
Fumagalli, F., Muschalik, M., H \"u llermeier, E., Hammer, B., and Herbinger, J. Unifying feature-based explanations with functional anova and cooperative game theory. arXiv preprint arXiv:2412.17152, 2024
2024 arXiv
-
[12]
A kernel statistical test of independence
Gretton, A., Fukumizu, K., Teo, C., Song, L., Sch \"o lkopf, B., and Smola, A. A kernel statistical test of independence. Advances in neural information processing systems, 20, 2007
2007
-
[13]
J., and Allesina, S
Grilli, J., Barab \'a s, G., Michalska-Smith, M. J., and Allesina, S. Higher-order interactions stabilize dynamics in competitive network models. Nature, 548 0 (7666): 0 210--213, 2017
2017
-
[14]
and Oja, E
Hyv \"a rinen, A. and Oja, E. Independent component analysis: algorithms and applications. Neural networks, 13 0 (4-5): 0 411--430, 2000
2000
-
[15]
and Ramdas, A
Kim, I. and Ramdas, A. Dimension-agnostic inference using cross u-statistics. Bernoulli, 30 0 (1): 0 683--711, 2024
2024
-
[16]
The Chi-Squared Distribution
Lancaster, H. The Chi-Squared Distribution. Wiley, 1969
1969
-
[17]
u gelgen, J., Park, J., Sch \
Laumann, F., Von K \"u gelgen, J., Park, J., Sch \"o lkopf, B., and Barahona, M. Kernel-based independence tests for causal structure learning on functional data. Entropy, 25 0 (12): 0 1597, 2023
2023
-
[18]
L., Laumann, F., Vallejo Mengod, S., and Barahona, M
Liu, Z., Peach, R. L., Laumann, F., Vallejo Mengod, S., and Barahona, M. Kernel-based joint independence tests for multivariate stationary and non-stationary time series. Royal Society Open Science, 10 0 (11): 0 230857, 2023 a
2023
-
[19]
L., Mediano, P
Liu, Z., Peach, R. L., Mediano, P. A., and Barahona, M. Interaction measures, partition lattices and kernel tests for high-order interactions. Advances in Neural Processing Systems, 36, 2023 b
2023
-
[20]
Liu, Z., Barahona, M., and Peach, R. L. Information-theoretic measures on lattices for high-order interactions. AISTATS - Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR, 2025
2025
-
[21]
Learning transferable features with deep adaptation networks
Long, M., Cao, Y., Wang, J., and Jordan, M. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp.\ 97--105. PMLR, 2015
2015
-
[22]
I., and Yosef, N
Lopez, R., Regier, J., Jordan, M. I., and Yosef, N. Information constraints on auto-encoding variational bayes. Advances in neural information processing systems, 31, 2018
2018
-
[23]
E., Peach, R., Mallas, E.-J., Rhodes, E., Laumann, F., Boyden, E
Luff, C. E., Peach, R., Mallas, E.-J., Rhodes, E., Laumann, F., Boyden, E. S., Sharp, D. J., Barahona, M., and Grossman, N. The neuron mixer and its impact on human brain dynamics. Cell Reports, 43 0 (6), 2024
2024
-
[24]
Lundberg, S. M. and Lee, S.-I. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 2017
2017
-
[25]
Kernel mean embedding of distributions: A review and beyond
Muandet, K., Fukumizu, K., Sriperumbudur, B., and Sch\" o lkopf, B. Kernel mean embedding of distributions: A review and beyond. Found. Trends Mach. Learn., 10 0 (1–2): 0 1–141, June 2017. ISSN 1935-8237. doi:10.1561/2200000060. URL https://doi.org/10.1561/2200000060
2017 doi
-
[26]
shapiq: Shapley interactions for machine learning
Muschalik, M., Baniecki, H., Fumagalli, F., Kolpaczki, P., Hammer, B., and H \"u llermeier, E. shapiq: Shapley interactions for machine learning. Advances in Neural Information Processing Systems, 37: 0 130324--130357, 2024
2024
-
[27]
Scikit-learn: Machine learning in P ython
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in P ython. Journal of ...
2011
-
[28]
M., Janzing, D., and Sch\" o lkopf, B
Peters, J., Mooij, J. M., Janzing, D., and Sch\" o lkopf, B. Causal discovery with continuous additive noise models. J. Mach. Learn. Res., 15 0 (1): 0 2009–2053, January 2014. ISSN 1532-4435
2009
-
[29]
u hlmann, P., Sch \
Pfister, N., B \"u hlmann, P., Sch \"o lkopf, B., and Peters, J. Kernel-based tests for joint independence. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80 0 (1): 0 5--31, 2018
2018
-
[30]
Copula-based kernel dependency measures
P\' o czos, B., Ghahramani, Z., and Schneider, J. Copula-based kernel dependency measures. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML'12, pp.\ 1635–1642, Madison, WI, USA, 2012. Omnipress. ISBN 9781450312851
2012
-
[31]
Consistency of permutation tests of independence using distance covariance, hsic and dhsic
Rindt, D., Sejdinovic, D., and Steinsaltz, D. Consistency of permutation tests of independence using distance covariance, hsic and dhsic. Stat, 10 0 (1): 0 e364, 2021
2021
-
[32]
E., Mediano, P
Rosas, F. E., Mediano, P. A., Rassouli, B., and Barrett, A. B. An operational information decomposition via synergistic disclosure. Journal of Physics A: Mathematical and Theoretical, 53 0 (48): 0 485001, 2020
2020
-
[33]
E., Mediano, P
Rosas, F. E., Mediano, P. A., Luppi, A. I., Varley, T. F., Lizier, J. T., Stramaglia, S., Jensen, H. J., and Marinazzo, D. Disentangling high-order mechanisms and high-order behaviours in complex systems. Nature Physics, 18 0 (5): 0 476--477, 2022
2022
-
[34]
E., Gutknecht, A., Mediano, P
Rosas, F. E., Gutknecht, A., Mediano, P. A., and Gastpar, M. Characterising high-order interdependence via entropic conjugation. arXiv preprint arXiv:2410.10485, 2024
2024 arXiv
-
[35]
K., Chwialkowski, K
Rubenstein, P. K., Chwialkowski, K. P., and Gretton, A. A kernel test for three-variable interactions with random processes. arXiv preprint arXiv:1603.00929, 2016
2016 arXiv
-
[36]
Higher-order organization of multivariate time series
Santoro, A., Battiston, F., Petri, G., and Amico, E. Higher-order organization of multivariate time series. Nature Physics, 19 0 (2): 0 221--229, 2023
2023
-
[37]
Higher-order connectomics of human brain function reveals local topological signatures of task decoding, individual identification, and behavior
Santoro, A., Battiston, F., Lucas, M., Petri, G., and Amico, E. Higher-order connectomics of human brain function reveals local topological signatures of task decoding, individual identification, and behavior. Nature Communications, 15 0 (1): 0 10244, 2024
2024
-
[38]
T., Benson, A
Schaub, M. T., Benson, A. R., Horn, P., Lippner, G., and Jadbabaie, A. Random walks on simplicial complexes and the normalized hodge 1-laplacian. SIAM Review, 62 0 (2): 0 353--391, 2020
2020
-
[39]
J., Bialek, W., et al
Schneidman, E., Still, S., Berry, M. J., Bialek, W., et al. Network information and connected correlations. Physical review letters, 91 0 (23): 0 238701, 2003
2003
-
[40]
A kernel test for three-variable interactions
Sejdinovic, D., Gretton, A., and Bergsma, W. A kernel test for three-variable interactions. Advances in Neural Information Processing Systems, 26, 2013 a
2013
-
[41]
Equivalence of distance-based and rkhs-based statistics in hypothesis testing
Sejdinovic, D., Sriperumbudur, B., Gretton, A., and Fukumizu, K. Equivalence of distance-based and rkhs-based statistics in hypothesis testing. The annals of statistics, pp.\ 2263--2291, 2013 b
2013
-
[42]
A permutation-free kernel two-sample test
Shekhar, S., Kim, I., and Ramdas, A. A permutation-free kernel two-sample test. Advances in Neural Information Processing Systems, 35: 0 18168--18180, 2022
2022
-
[43]
A permutation-free kernel independence test
Shekhar, S., Kim, I., and Ramdas, A. A permutation-free kernel independence test. Journal of Machine Learning Research, 24 0 (369): 0 1--68, 2023
2023
-
[44]
Sloane, N. J. et al. The on-line encyclopedia of integer sequences, sequence a000296. Published electronically at https://oeis. org, 2018
2018
-
[45]
Feature selection via dependence maximization
Song, L., Smola, A., Gretton, A., Bedo, J., and Borgwardt, K. Feature selection via dependence maximization. Journal of Machine Learning Research, 13 0 (5), 2012
2012
-
[46]
Lancaster interactions revisited
Streitberg, B. Lancaster interactions revisited. The Annals of Statistics, pp.\ 1878--1885, 1990
1990
-
[47]
The low-rank hypothesis of complex systems
Thibeault, V., Allard, A., and Desrosiers, P. The low-rank hypothesis of complex systems. Nature Physics, 20 0 (2): 0 294--302, 2024
2024
-
[48]
Young, J.-G., Petri, G., and Peixoto, T. P. Hypergraph reconstruction from network data. Communications Physics, 4 0 (1): 0 1--11, 2021
2021
-
[49]
Kernel-based conditional independence test and application in causal discovery
Zhang, K., Peters, J., Janzing, D., and Sch\" o lkopf, B. Kernel-based conditional independence test and application in causal discovery. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI'11, pp.\ 804–813, Arlington, Virginia, USA, ...
2011
-
[50]
Large-scale kernel methods for independence testing
Zhang, Q., Filippi, S., Gretton, A., and Sejdinovic, D. Large-scale kernel methods for independence testing. Statistics and Computing, 28: 0 113--130, 2018
2018
-
[51]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.