Pith. sign in

REVIEW 3 major objections 4 minor 26 references

Independence Testing for Temporal Data

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces DCorrX and MGCX, block-permutation tests of pairwise independence between stationary time series at all lags up to a maximum M, and proves they are asymptotically valid and consistent while also estimating the lag of…

desk verdict Useful, clearly written extension of MGC/DCorr to temporal data with real code, but Theorem 3's block-permutation validity proof has a genuine one-sided-mixing gap and the abstract overclaims. read the letter →

arxiv 1908.06486 v5 pith:JIET37PV submitted 2019-08-18 stat.ML cs.LGstat.ME

Cencheng Shen , Jaewon Chung , Ronak Mehta , Ting Xu , Joshua T. Vogelstein This is my paper · ORCID
classification stat.MLcs.LGstat.ME MSC 62G1062M1062H20
keywords temporalindependencetestingdistancecorrelationmultiscalegraphblockpermutationtimeseriesnonparametrichypothesisoptimallagestimationuniversalconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether two stationary time series are related at any lag from 0 to M, without assuming linearity, normality, or multiple-test corrections. It claims that summing the distance correlation (or its multiscale variant) over the lags, weighting by (n-j)/n, and calibrating the p-value by permuting blocks of consecutive observations instead of individual points, yields an asymptotically valid and consistent test of the joint null hypothesis. The same procedure also consistently estimates the lag at which dependence is strongest. If correct, this turns any i.i.d. distance- or kernel-based dependence measure into a practical tool for temporal data, including high-dimensional nonlinear settings where classical cross-correlation and Ljung-Box tests can fail.

What carries the argument

The central object is the block permutation null, built by cutting the Y series into consecutive blocks of size b_n, permuting the blocks, and recomputing the test statistic; unlike a full permutation, this keeps nearby observations together and imitates the serial dependence of the original series. It is coupled to cross-lag distance correlation: DCorrX(j) = DCorr(X_t, Y_{t-j}) weighted by (n-j)/n and summed over j = 0..M, while MGCX is the multiscale version that takes a smoothed maximum over local scales at each lag. Under the weak-dependence assumption, observations separated by many lags are nearly independent, so both the sample statistic converges to its population value and the block-permuted Y blocks behave approximately like independent copies of Y.

What would settle it

Simulate two independent stationary AR(1) series with phi = 0.8, run DCorrX and MGCX with block permutation at $\alpha$ = 0.05 for n = 200, 500, and 2000 with block size roughly $\sqrt$(n); Theorem 3 predicts the rejection frequency converges to 0.05, and if it converges anywhere else the validity claim fails. To test whether the assumptions are necessary, repeat with independent long-range dependent fractional Gaussian noise with Hurst exponent H = 0.85: if the rejection frequency stays at 0.05 despite Assumption 4 failing, that condition is not doing the claimed work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that block permutation can repair the invalidity of ordinary permutation tests for dependent data, so that a lag-summed distance correlation test and its multiscale version become legitimate omnibus tests of temporal independence. Under strict stationarity, finite second moments, and a uniform weak-dependence condition (Assumption 4), Theorem 3 shows the block-permutation p-value controls Type I error asymptotically, Theorem 4 shows power goes to 1 under any fixed alternative with dependence at some lag no greater than M, and Theorem 5 shows the estimated lag of maximal dependence is consistent. The proof works because block permutation preserves the short-range serial dependence of Y while making the shifted blocks asymptotically independent of X, so the permuted replicates approximate the null distribution.

Load-bearing premise

The procedure only works when the two series are stationary and their dependence decays fast enough with lag: if either series has long-range memory or a trend, the block-permuted p-values are no longer guaranteed to approximate the null distribution.

Editorial extensions

If this is right

  • Researchers can replace separate lag-by-lag tests with Bonferroni corrections by one omnibus test for dependence within a window of M lags.
  • The procedure returns a consistent estimate of the lag at which dependence is strongest, so it identifies not only whether two series interact but at what temporal offset.
  • Because the result applies to any metric or characteristic kernel of the required type, the block-permutation construction extends distance correlation, multiscale graph correlation, and kernel measures such as HSIC to temporal data.
  • In high-dimensional, low-sample settings where linear cross-correlation is blind, MGCX retains power and also reports the geometric scale of the relationship, as demonstrated on brain-imaging time series.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the same block-permutation wrapper should calibrate any i.i.d. independence statistic that vanishes only under independence, so the validity proof likely transfers to a broader family of temporal tests beyond the two demonstrated.
  • Editorial inference: the theorems keep the maximum lag M fixed as n grows, so using M that grows with sample size would require new rate conditions before the omnibus p-value can be trusted; practitioners should treat M as a small, domain-chosen window.
  • Editorial inference: the optimal-scale pair returned by MGCX at the estimated optimal lag can serve as a screening diagnostic for functional connectivity, with scale (1,1) flagging effectively linear coupling and other scales flagging local or nonlinear coupling.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This manuscript proposes DCorrX and MGCX, omnibus nonparametric tests of independence between two strictly stationary time series at lags 0 through M. The test statistic is a weighted sum of lagged distance-correlation or multiscale-graph-correlation estimates, and the null distribution is estimated by permuting blocks of one series rather than individual observations. The authors claim asymptotic validity of the block-permutation procedure under the null (Theorem 3), consistency under fixed alternatives (Theorem 4), and consistency of the optimal-lag estimator (Theorem 5), and they support the method with simulations and an fMRI resting-state connectivity analysis. The main theoretical results are stated in Section 4 with proofs collected in the Appendix.

Significance. The methodological goal is valuable: a nonparametric, multivariate, nonlinear omnibus test for temporal dependence that returns a single p-value and also characterizes the lag and geometric scale of the dependence would be a practically useful addition to the time-series toolbox. The paper builds on the established DCorr/MGC framework, ships open-source code in the hyppo package, compares against ShiftHSIC, WildHSIC, and Ljung-Box variants, and demonstrates the method on HCP fMRI data. However, the central validity theorem is not established under the stated assumptions: Assumption 4 is one-sided while the proof treats permuted lags symmetrically, and the counterexample in my report shows that Theorem 3 as stated is false. Because the proof gap is local and a two-sided mixing condition would plausibly repair it, the contribution is significant but currently not rigorous enough for publication.

major comments (3)
  1. [Section 4 (Assumption 4) and Appendix proof of Theorem 3] Assumption 4 is one-sided: it controls only sup|F_{X_t,Y_{t-j}} - F_{X_t}F_{Y_{t-j}}| as j goes to infinity, i.e., X_t against past values of Y. Block permutation shifts contiguous blocks of Y, so for a large fraction of positions t the permuted index pi(t) differs from t by a nonzero multiple of the block size b_n, equally often in the positive direction. Thus the permuted series pairs X_t with future values Y_{t+b_n}, whose dependence on X_t is not controlled by Assumption 4. The proof's condition P(|t-pi(t-j)| < gamma for some |j| <= M) treats positive and negative j symmetrically, but no assumption bounds F_{X_t,Y_{t+j}} as j grows. Concretely, let Y_t and epsilon_t be iid standard Gaussian and define X_t = c * sum_{k>=1} k^{-3/4} Y_{t+k} + epsilon_t. This process is strictly stationary, has finite second moments, satisfies Assumptions 1, 2, 4, and 5, and satisfies H0 for every fixed M because X_t is independent of Y_t and of all Y_{t-j}, j>=1. If block permutation shifts a block by +b_n, then for most t the permuted pair is (X_t, Y_{t+b_n}), whose distance correlation has size about c b_n^{-3/4}; the permuted statistic then has positive mean while the observed statistic estimates zero, so p-values concentrate near 1 and the rejection rate tends to 0 rather than alpha. The final step of the proof also invokes the continuous mapping theorem without establishing the required joint convergence of the permuted U-statistic. Theorem 3 is therefore not a consequence of the stated assumptions.
  2. [Appendix, proof of Theorem 1] The proof fixes gamma for a given epsilon and then treats index pairs with |i-j| > gamma as approximately independent, carrying an O(epsilon) error. Because epsilon is held fixed while n grows, Theorem 1 as proven only gives E[DCov_{k,l}^n] = DCov_{rho_k,rho_l} + O(1/n) + O(epsilon) and Var[DCov_{k,l}^n] = O(1/n) + O(epsilon), which is not a convergence statement. The argument needs an n-dependent gamma_n with gamma_n -> infinity, gamma_n/n -> 0, and an approximation error that tends to 0; otherwise Theorem 2 (convergence in probability) and hence Theorem 4 are not established. The appeal to Theorem 5 of Shen et al. applies to i.i.d. data; with |i-j| > gamma the pairs are only approximately independent, so the approximation error must be shown to vanish.
  3. [Section 4, Theorem 5] The proof of optimal-lag consistency appeals only to the finiteness of the search space and pointwise convergence of the sample statistics. If the population maximum M* = argmax_{0<=j<=M} DCorr(j) is not unique, or if there are near-ties whose separation does not grow with n, the sample argmax need not converge to a single M*. Please state explicitly that M* is the unique maximizer, or reformulate the theorem as convergence to the set of maximizers.
minor comments (4)
  1. [Section 2.4] The displayed formula for DCov(X,Y) is actually the squared distance covariance; the square should be applied consistently, or the notation should be changed to DCov^2.
  2. [Section 3, block permutation step 2] The example writes '(Y36,Y35,...,Y40)', but the order within a block should be preserved; it should read '(Y36,Y37,...,Y40)'.
  3. [Section 5.2] The sentence 'DCorrX and MGCX always estimate the correct lag' is too strong for finite n; it should be rephrased as 'with probability tending to 1 as n grows'.
  4. [Section 2.3] There is a typo: 'empircal' should be 'empirical'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the lag-averaged block-permutation test is a genuine extension of prior distance-correlation/MGC theory, not a restatement of its inputs.

full rationale

The paper's central claims (Theorems 3-5) are not forced by the definitions or fitted inputs. The test statistics DCorrX and MGCX are constructed as explicit lag-averaged sums of DCorr/MGC statistics, and the block-permutation null is a distinct resampling procedure; neither quantity is defined in terms of the p-value or the optimal-lag estimate it is said to predict. Theorem 5's optimal-lag estimator is a standard argmax consistency claim, not a fitted parameter renamed as a prediction. The proof does lean on Shen et al. [20] for the i.i.d. local distance-correlation bias/variance, which is a self-citation, but it is not circular under the stated criteria: it is a published, externally checkable theorem whose assumptions do not include the temporal block-permutation result, and the new argument adapts it using Assumption 4 rather than assuming the conclusion. A possible mathematical gap exists in Theorem 3's proof, since Assumption 4 only controls FXt,Yt-j for j→∞ (past lags) while block permutation can align X_t with future Y values; however, that is a correctness/validity concern rather than a circularity concern, because it concerns whether the stated assumptions imply the theorem, not whether the theorem is definitionally identical to its inputs. The paper also explicitly notes unaddressed extensions (growing M, consistency against all alternatives) in Section 7, which further indicates the claims are bounded, not circular.

Assumptions & free parameters 2 free parameters · 7 assumptions · 0 invented entities

The method rests on stationarity and mixing assumptions (Assumptions 1-5) and on prior distance-correlation/MGC theorems. The only user-chosen numbers are the maximum lag M and block size b_n; no constants are fitted to data to obtain the claimed results.

free parameters (2)
  • M (maximum lag)
    User-selected hyperparameter; the test is defined only for lags 0..M, and consistency is proven for fixed M.
  • b_n (block size)
    Chosen by the user for block permutation; Assumption 3 requires b_n→∞ and b_n/n→0, but no data-dependent choice is given.
assumptions (7)
  • domain assumption The joint process {(X_t,Y_t)} is strictly stationary.
    Defines DCorr(j) as independent of t; used in Section 2.2 and Assumption 1.
  • domain assumption Finite second moments for X_t and Y_t.
    Needed for distance covariance to be well-defined; Assumption 2.
  • domain assumption Block size b_n→∞, b_n/n→0, and permutation count R_n→∞.
    Needed for the block permutation null approximation; Assumption 3.
  • domain assumption Weak dependence: sup|F_{Xt,Yt-j} - F_Xt F_Yt-j|→0 as j→∞, and similar for marginals.
    Critical for both consistency of DCov_n and validity of block permutation; Assumption 4.
  • domain assumption The process is nonconstant and continuous for all t.
    Ensures DCov normalization denominators are nonzero; Assumption 5.
  • standard math MGC local correlation consistency and distance-kernel equivalence results from prior work hold.
    The paper uses Theorem 5 of Shen et al. 2020 and the equivalence results of Sejdinovic et al. 2013 and Shen and Vogelstein 2019 without reproof.
  • standard math Distance covariance characterizes independence: DCorr=0 iff independent.
    Used to argue consistency under the alternative; cited from Szekely et al. 2007.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Independence Testing for Temporal Data." pith.science (2026). https://pith.science/paper/JIET37PV

@misc{pith2026190806486,
  author       = {Pith},
  title        = {Pith review of: Independence Testing for Temporal Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JIET37PV}},
  note         = {Machine review of arXiv:1908.06486}
}
read the original abstract

Temporal data are increasingly prevalent in modern data science. A fundamental question is whether two time series are related or not. Existing approaches often have limitations, such as relying on parametric assumptions, detecting only linear associations, and requiring multiple tests and corrections. While many non-parametric and universally consistent dependence measures have recently been proposed, directly applying them to temporal data can inflate the p-value and result in an invalid test. To address these challenges, this paper introduces the temporal dependence statistic with block permutation to test independence between temporal data. Under proper assumptions, the proposed procedure is asymptotically valid and universally consistent for testing independence between stationary time series, and capable of estimating the optimal dependence lag that maximizes the dependence. Moreover, it is compatible with a rich family of distance and kernel based dependence measures, eliminates the need for multiple testing, and exhibits excellent testing power in various simulation settings.

Figures

Figures reproduced from arXiv: 1908.06486 by the authors.

Figure 1
Figure 1. Multivariate but Independent Auto-Regressive Simulation. AR(1) simulations of the multivariate process as in Eq 5.1 with 300 simulations. Left AR coefficient φ = 0.5 with 1000 permutation replicates. Right Sample size is N = 1200 with 100 permutation replicates. The black dashed line represents α = 0.05. These simulations support the claim of asymptotic validity of each approach in this independence setting. Extinct… view at source ↗
Figure 2
Figure 2. Dependent Auto-Regressive Linear and Nonlinear Simulations. 300 simulations are run, each with 1000 permutation replicates, with the percent of series for which each test rejects the null hypothesis recorded. The black dashed line represents α = 0.05. Left Process 5.2: Ljung-Box, DCorrX, and MGCX are consistent with rates ranked in that order. Right Process 5.3: MGCX and DCorrX are both consistent, with MGCX showing… view at source ↗
Figure 3
Figure 3. Extinct Gaussian Nonlinear Dependence Simulations 300 simulations of the process in Equation 5.4 are run, each with 100 permutation replicates and 1200 samples, with the percent of series for which each test rejects the null hypothesis recorded. The coefficient φ = 0.2 and the radius r = 1. The extinction rate δ is shown on the horizontal axis. The black dashed line represents α = 0.05. MGCX shows the highest sensit… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Distribution of Optimal Lag Estimates The colored bar above lag j represents the empirical frequency of Mˆ = j, with red being MGCX and purple being DCorrX with M = 10 and 100 trials. Top Process 5.5: The correlated AR(3) process has true cross-correlation as a measure…
Figure 5
Figure 5. Figure 5: Top Left For the visualized matrix Q, Quv is the resulting optimal lag of MGCX applied to {X(u) } and {X(v)} with maximum lag M = 10. Dark blue values represent large values of optimal lag, which imply that {X(u)} depends strongly on past values of {X(v)}. Top Right Fo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 25 canonical work pages

  1. [1]

    A kernel independence test for random processes

    Kacper Chwialkowski and Arthur Gretton. A kernel independence test for random processes. International Conference on Machine Learning, pages 1422–1430, 2014

    work page 2014

  2. [2]

    A wild bootstrap for degenerate kernel tests

    Kacper P Chwialkowski, Dino Sejdinovic, and Arthur Gretton. A wild bootstrap for degenerate kernel tests. Advances in neural information processing systems, pages 3608–3616, 2014

    work page 2014

  3. [3]

    Cryer and Kung sik Chan

    Jonathan D. Cryer and Kung sik Chan. 14.9: Other methods of spectral estimation.â ˘A˙I time series analysis with applications in r. Springer, 2011

    work page 2011

  4. [4]

    Optimal block size for variance estimation by a spatial block bootstrap method

    Nordman D, S Lahiri, and B Fridley. Optimal block size for variance estimation by a spatial block bootstrap method. SankhyÄ ˛ A:The Indian Journal of Statistics, 2007

    work page 2007

  5. [5]

    An updated literature review of distance correlation and its applications to time series

    Dominic Edelmann, Konstantinos Fokianos, and Maria Pitsillou. An updated literature review of distance correlation and its applications to time series. International Statistical Review, 2018

    work page 2018

  6. [6]

    Consistent testing for pairwise dependence in time series

    K Fokianos and M Pitsillou. Consistent testing for pairwise dependence in time series. Technometrics, 59(2), 2017

    work page 2017

  7. [7]

    A multi-modal parcellation of human cerebral cortex

    Glasser et al. A multi-modal parcellation of human cerebral cortex. Nature, 2016

    work page 2016

  8. [8]

    Gretton and L

    A. Gretton and L. Gyorfi. Consistent nonparametric tests of independence. Journal of Machine Learning Research, 11:1391–1423, 2010

    work page 2010

Show all 26 references
  1. [9]

    Gretton, R

    A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Scholkopf. Kernel methods for measuring independence. Journal of Machine Learning Research, 6:2075–2129, 2005

    2005

  2. [10]

    A thinned block bootstrap variance estimation procedure for inhomo- geneous spatial point patterns

    Y Guan and J Meng. A thinned block bootstrap variance estimation procedure for inhomo- geneous spatial point patterns. Journal of the American Statistical Association, 2007. doi: 10.1198/016214507000000879

    2007 doi

  3. [11]

    New hsic-based tests for independence between two stationary multivariate time series

    Wang Guochang, Wai Keung Li, and Ke Zhu. New hsic-based tests for independence between two stationary multivariate time series. ArXiv E-prints, 2018

    2018

  4. [12]

    Hypothesis testing in time series via the empirical characteristic function: A generalized spectral density approach

    Y ongmiao Hong. Hypothesis testing in time series via the empirical characteristic function: A generalized spectral density approach. Journal of the American Statistical Association, 94(448): 1201, 1999. doi: 10.2307/2669935

    1999 doi

  5. [13]

    Y . Lee, C. Shen, C. E. Priebe, and J. T. Vogelstein. Network dependence testing via diffusion maps and distance-based correlations. Biometrika, 106(4):857–873, 2019

    2019

  6. [14]

    Mø sri and Gà ˛ abor J

    Tamà ˛ as F . Mø sri and Gà ˛ abor J. SzÃl’kely. Four simple axioms of dependence measures. Metrika: International Journal for Theoretical and Applied Statistics, 82(1):1–16, January 2019. doi: 10.1007/s00184-018-0670-3 . URL https://ideas.repec.org/a/spr/metrik/v82y2019i1d10...

    2019 doi

  7. [15]

    Panda, S

    S. Panda, S. Palaniappan, J. Xiong, A. Swaminathan, S. Ramachandran, E. Bridgeford, C. Shen, and J. Vogelstein. Mgc: Comprehensive high dimensional independence testing python package. https://arxiv.org/abs/1907.02088, 2019

    1907 arXiv

  8. [16]

    The impact of bootstrap methods on time series analysis

    Dimitris N Politis. The impact of bootstrap methods on time series analysis. Statist. Sci., 18(2): 219–230, 2003

    2003

  9. [17]

    Sejdinovic, B

    D. Sejdinovic, B. Sriperumbudur, A. Gretton, and K. Fukumizu. Equivalence of distance-based and 18 rkhs-based statistics in hypothesis testing. Annals of Statistics, 41(5):2263–2291, 2013

    2013

  10. [18]

    Shen and J

    C. Shen and J. T. Vogelstein. The exact equivalence of distance and kernel methods in hypothesis testing. https://arxiv.org/abs/1806.05514, 2019

    2019 arXiv

  11. [19]

    C. Shen, C. E. Priebe, and J. T. Vogelstein. The exact equivalence of independence testing and two-sample testing. https://arxiv.org/abs/1910.08883, 2019

    1910 arXiv

  12. [20]

    C. Shen, C. E. Priebe, and J. T. Vogelstein. From distance correlation to multiscale graph correla- tion. Journal of the American Statistical Association, 115(529):280–291, 2020

    2020

  13. [21]

    Szekely and M

    G. Szekely and M. Rizzo. Brownian distance covariance. Annals of Applied Statistics, 3(4):1233– 1303, 2009

    2009

  14. [22]

    Szekely, M

    G. Szekely, M. Rizzo, and N. Bakirov. Measuring and testing independence by correlation of distances. Annals of Statistics, 35(6):2769–2794, 2007

    2007

  15. [23]

    J. T. Vogelstein, Q. Wang, E. Bridgeford, C. E. Priebe, M. Maggioni, and C. Shen. Discovering and deciphering relationships across disparate data modalities. eLife, 8:e41690, 2019

    2019

  16. [24]

    approximately independent

    J Zhu and G Morgan. Comparison of spatial variables over subregions using a block bootstrap. Journal of Agricultural, Biological, and Environmental Statistics, 2003. doi: https://doi.org/10. 1198/1085711043154. 19 Appendix. The assumptions and results are listed once again bel...

    2003

  17. [25]

    For Rn large, by the Glivenko-Cantelli theorem, we can have P [| ˆFRn,Tn(s)−FTn| > ϵ] < δ

  18. [26]

    DCorrX and MGCX with block permutation are asymptotically valid tests

    Therefore, by letting the number Rn of resampled series (Yπ(t)) grow with n, say Rn = n, the estimate of the distribution of Tn is consistent, implying that P (T∗ n≥Tn,K,α)→α. DCorrX and MGCX with block permutation are asymptotically valid tests. Theorem 4. Under the alternati...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.