Pith. sign in

REVIEW 2 major objections 5 minor 21 references

Overlapping window tests for correlation and trend

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sliding-window tests: new information is the eigenvalue-one component

desk verdict A coherent and useful spectral framework for overlapping-window tests, with a real factor-of-two error in the printed normalization formula that needs fixing before the results work as stated. read the letter →

arxiv 2607.13173 v1 pith:Z67CDFUV submitted 2026-07-14 math.ST stat.TH

Abbas Alhakim This is my paper
classification math.STstat.TH MSC 62G1062G2062M1060J05
keywords sliding-windowstatisticsoverlappingblocksMarkovchaincovarianceoperatorspectraldecompositionincrementalinformationcorrelationdetectiontrendlocalasymptotics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that for any centered sliding-window statistic computed from independent observations, the asymptotic variance is governed by a covariance operator whose eigenvalues are exactly the integers 0 through k. That spectral structure splits the space of window functions into orthogonal components tied to overlap level, and the component at eigenvalue 1 is precisely the information genuinely added when the window is enlarged from k-1 to k. The paper supplies a constructive algorithm—repeated integration over boundary variables—that extracts these components without knowing the operator's eigenvectors, plus explicit variance formulas for normalizing the resulting statistics. It then applies the machinery to correlation and trend detection, deriving the local detection scales (n^{-1/4} for quartic correlation statistics, n^{-1/2} for a rank-based trend statistic) and a relative incremental index that declines like k^{-3} in the worked examples. A careful reader would care because the framework turns window-size choice from an ad hoc tuning decision into a question with a spectral answer.

What carries the argument

The covariance operator B_k = I + P + P* + ... + P^{k-1} + (P*)^{k-1}, where P is the transition operator of the overlapping-window Markov chain Y_i = (X_i,...,X_{i+k-1}); its integer spectrum and eigenspaces L_l are the backbone of the decomposition. The auxiliary spaces S_l, consisting of sums of l functions supported on consecutive blocks of length k-l+1 with side-centering conditions, provide a constructive route to the eigenspaces: the paper's main algorithm (Theorem 3.1) obtains s_l ∈ S_l by telescoping marginal integrals of f, and for symmetric f the s_l are automatically eigenfunctions. The identity L_1 = W_{k-1}^⊥, characterized by vanishing conditional expectations on the first and

What would settle it

Compute the transition operator P for the overlapping-window chain on a small finite alphabet (e.g., binary states, k=3), form B_3 = I + P + P* + P^2 + (P*)^2, and check numerically whether its nonzero eigenvalues are exactly 1, 2, 3 with the claimed eigenspace decomposition; any mismatch would collapse the framework. Alternatively, pick a non-symmetric f, compute σ²(f) directly from the long-run covariance formula Var(f(Y_1)) + 2 Σ_{r=1}^{k-1} Cov(f(Y_1), f(Y_{1+r})), and compare it with Σ l‖f_l‖² obtained from the paper's auxiliary decomposition—disagreement would refute the spectral varianc

Watch

Extended reading notes

Core claim

The central claim is that the covariance operator B_k of the overlapping-window Markov chain has spectrum {0,1,...,k}, so every centered window function f decomposes uniquely as f = f_0 + f_1 + ... + f_k with f_l in the eigenvalue-l eigenspace L_l, and the asymptotic variance equals sum_{l=1}^k l ||f_l||^2. The paper identifies the incremental information introduced by enlarging the window to size k as the projection of f onto L_1, and proves that L_1 is exactly the orthogonal complement of W_{k-1}, the space of functions representable from k-1 consecutive observations; equivalently, f is incremental iff its left and right conditional expectations vanish. For symmetric polynomial correlation

Load-bearing premise

Everything rests on Theorem 2.3, quoted without proof from an earlier paper, that the covariance operator B_k has eigenvalues exactly 0,1,...,k and that each auxiliary space S_l splits as the eigenspace L_l plus a kernel component.

Editorial extensions

If this is right

  • Every centered window statistic can be normalized by the variance formula σ²(f) = Σ_{l=1}^k l‖f_l‖², and the components f_l can be computed by the paper's constructive marginal-integration algorithm.
  • The incremental part f_inc = Proj_{L_1} f isolates the dependence structure that first appears at span k; statistics built purely from endpoint interactions (x_1-μ)(x_k-μ) g(x_2,...,x_{k-1}) lie entirely in L_1 and need no projection.
  • The relative incremental index I_k(f) = ‖f_inc‖²/σ²(f) gives a window-size selection rule; for both the polynomial correlation family and the rank-based trend family it decreases in k, asymptotically as k^{-3}, so beyond a certain scale larger windows add little genuinely new relative information.
  • Local detection rates follow from the lowest-order nonvanishing term of the mean under the alternative: the quartic correlation statistics are second-order (local scale n^{-1/4}), while the trend statistic K is first-order (scale n^{-1/2}), and simulations confirm these scalings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the spectral decomposition is as clean as claimed, the same L_1-vs-W_{k-1} dichotomy could be used to build pure incremental tests for long-range lags without contamination from lower-span structure; the endpoint-interaction form gives a template that could be adapted to other dependence alternatives.
  • The paper's own remark that the projected trend statistic K_1 lacks a Pitman regime suggests a general phenomenon: isolating the incremental component removes the aggregation that produces standard local asymptotics, so pure incremental statistics may need different distributional tools; this is an open question the author flags.
  • The relative incremental index, though introduced for comparing nested window sizes, could equally serve as a metric for comparing different statistics at the same window, and its monotone decay in the examples hints at a broader principle about how information saturates with window length; a general theorem on decay rates is left as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper develops a spectral framework for overlapping sliding-window statistics under an i.i.d. null. Each window statistic f is seen as an element of L^2(X^k), and the asymptotic variance is represented as <B_k f,f> for an overlap covariance operator B_k. The paper quotes a spectral theorem stating that B_k has eigenvalues 0,1,...,k, yielding an orthogonal decomposition H_k = L_0 ⊕ ... ⊕ L_k. The eigenvalue-one subspace L_1 is identified with the incremental dependence information introduced by enlarging the window, and an auxiliary decomposition is constructed to compute the projection onto L_1. This is applied to symmetric polynomial correlation statistics (elementary symmetric polynomials) and to a localized rank trend statistic K, with explicit variance formulas, a relative incremental index I_k(f), local asymptotic calculations, and simulations. The central claim is that every centered sliding-window statistic can be separated into overlap-level components and that the L_1 component captures exactly the new information added by increasing the window size.

Significance. If the framework is correct, it gives a systematic, constructive method for designing and normalizing overlapping-window tests, and it provides a natural quantitative notion of incremental information. The spectral decomposition and the explicit projection procedure are elegant and potentially useful for a broad class of statistics. The paper also contains concrete, checkable formulas, and the simulation study illustrates the predicted local detection scales. However, the practical payoff depends on the correctness of the explicit variance formulas, and Proposition 4.2 contains a concrete factor-of-two error. This is a load-bearing defect: the normalization of the proposed g_1 statistic is wrong as printed, and it is inconsistent with Proposition 4.8. The central structural claim about the eigenvalue-one decomposition is not refuted, and the error appears fixable, but the manuscript cannot be accepted in its current form.

major comments (2)
  1. [§4.2, Proposition 4.2] The stated asymptotic variance for g_1 is off by a factor of 2. For k=4, μ=0, σ^2=1, g_1 = X1X2X3X4 and ||g_1||^2 = 1; since g_1 ∈ L_1, the spectral formula gives σ^2(g_1) = 1. The proposition prints (k-2)(k-3)/4 = 1/2. Tracing the proof, the count E[S^2] = C(k-2,2)α^2 + 2(k-2)C(k-3,2)αμ^2 + 6C(k-2,4)μ^4 is correct, but when it is multiplied by σ^4 the factor C(k-2,2) = (k-2)(k-3)/2 is replaced by (k-2)(k-3)/4, losing a factor of 2. The corrected formula is σ^2(g_1) = σ^4 C(k-2,2)[(μ^2+σ^2)^2 + 2(k-4)μ^2(μ^2+σ^2) + (k-4)(k-5)/2 μ^4].
  2. [§4.4, Proposition 4.8] Proposition 4.8 states that under N(0,1), σ^2(g_1) = C(k-2,2). This is the correct value (it equals ||g_1||^2), but it is inconsistent with Proposition 4.2, which gives half that value. The text says the identity follows from Proposition 4.2, which cannot be true. After correcting Proposition 4.2, the two statements become consistent, but as printed the normalization formulas contradict each other. This also calls into question the simulation claim of correct calibration, since the simulation section says the variance formula of Proposition 4.2 is used for normalization.
minor comments (5)
  1. [§2.3, Theorem 2.3] Theorem 2.3, the spectral theorem for B_k, is quoted from the author's earlier paper [3] without proof. Citing a published result is acceptable, but since the entire framework depends on it, a short proof or a more self-contained statement in an appendix would strengthen the paper.
  2. [§4.2, proof of Proposition 4.2] The final sentence 'Multiplying by σ^4 and replacing α gives the stated formula' hides the algebraic error described in the major comments. The simplification step should be written out explicitly.
  3. [§5, simulations] The simulation section asserts that all statistics are 'correctly calibrated under the null.' If the normalization used the printed Proposition 4.2, this cannot hold for g_1. The authors should either state which variance formula was actually used or re-run the simulations with the corrected formula.
  4. [§4.2, Remark on K_1] The statement that K_1 'does not appear to admit a standard Pitman-type local asymptotic regime' is based on simulations rather than a proof. The wording should distinguish an empirical observation from a proven fact.
  5. [§2.2, operator definition] The adjoint P^* is used but not explicitly defined in the text. Defining it would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the headline 'incremental information = eigenvalue-one component' is true by definition; the independent content is the L1 = W^perp identification.

  1. self definitional [Section 1 (Introduction) and Definition 3.5, Section 3.2]
    "A central structural outcome is that the incremental information is exactly the eigenvalue-one component of the asymptotic covariance operator. ... Definition 3.5. (Incremental Dependence Information) For f∈H_k, the incremental dependence information at window size k is defined as the orthogonal projection f_inc := Proj_{L1}f."

    The paper's headline 'outcome' is not derived: Definition 3.5 stipulates that 'incremental dependence information' means Proj_{L1} f, and Theorem 2.3 identifies L1 as the eigenvalue-one eigenspace. Hence the statement 'incremental information is exactly the eigenvalue-one component' is true by construction. The substantive, non-circular content is Proposition 3.6, showing L1 = W^\perp_{k-1} (the orthogonal complement of smaller-window statistics); that proposition is independent of the definitional labeling and provides the real interpretation.

full rationale

The rest of the derivation chain is not circular. Theorem 3.1 gives a constructive auxiliary decomposition with a proof; Propositions 4.1, 4.2, and 4.5 are direct computations; the spectral facts in Theorem 2.3 and Section 2.3 are imported from the author's prior published work ([1],[3]) as parameter-free theorems with stated assumptions that do not include the present target, so under the stated rules they are real external evidence rather than self-citation circularity. The factor-of-two discrepancy between Proposition 4.2 and Proposition 4.8's N(0,1) normalization is an arithmetic/correctness problem, not a circularity, and does not affect this score. The self-definitional labeling of 'incremental information' is a genuine but partial circularity because the paper presents it as a 'central structural outcome'; the underlying mathematical content (L1 = W^\perp) is independently proved.

Assumptions & free parameters 3 free parameters · 3 assumptions · 2 invented entities

The framework does not fit any parameters to data; all its inputs are user-chosen design choices such as window size, polynomial degree, and lag weights. It relies on standard Markov-chain central limit theory and, more substantially, on the spectral theorem for B_k proved in the author's earlier papers. No new physical entities are postulated.

free parameters (3)
  • window size k = user-chosen
    k defines the Hilbert space H_k and all decompositions; chosen by hand in examples and simulations, and not estimated from data.
  • degree l of elementary symmetric polynomial g = user-chosen
    For the correlation detector, l is the degree of the monomial products; it directly affects the variance, the incremental component, and the local scaling.
  • trend weight coefficients alpha_delta = user-chosen, e.g. (1,1,1)
    The weights in the trend statistic K control which lags are emphasized; the asymptotic variance and the incremental index both depend on them, but they are not learned from data.
assumptions (3)
  • domain assumption The overlapping-window process Y_i forms a time-homogeneous Markov chain on X^k with transition operator P, and the asymptotic variance of S_f is <B_k f, f> with B_k = I + P + P* + ... .
    Invoked in Section 2.2 as a standard long-run variance formula for additive functionals of Markov chains, citing [6,15]; foundational for all later variance computations.
  • domain assumption The covariance operator B_k has spectrum exactly {0,1,...,k} and the auxiliary spaces satisfy S_l = L_l + (S_l cap ker B_k).
    Quoted as Theorem 2.3 from the author's own prior paper [3, Theorem 2] and attributed to [1,3]; not proved in the present manuscript. The whole orthogonal decomposition depends on it.
  • domain assumption Under the i.i.d. null, S_f(n) satisfies a central limit theorem with asymptotic variance sigma^2(f).
    Needed for all the asymptotic normality claims and local power limits; taken from Gordin-Lifshits CLT theory [6,15] and [3], not derived in the paper.
invented entities (2)
  • Incremental dependence information f_inc = Proj_{L1} f
    purpose: To isolate the dependence structure introduced uniquely at window size k, as opposed to structure already available from smaller windows.
    This is a mathematical construct defined by orthogonal projection; it has no externally falsifiable handle outside the paper. Its properties follow from the spectral decomposition.
  • Relative incremental index I_k(f) = ||f_inc||^2 / sigma^2(f)
    purpose: To quantify how much of a statistic's asymptotic variance is attributable to the full-window incremental component, and to guide scale selection.
    A dimensionless ratio of norms; well-defined within the framework but not an independent observable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Overlapping window tests for correlation and trend." pith.science (2026). https://pith.science/paper/Z67CDFUV

@misc{pith2026260713173,
  author       = {Pith},
  title        = {Pith review of: Overlapping window tests for correlation and trend},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z67CDFUV}},
  note         = {Machine review of arXiv:2607.13173}
}
read the original abstract

We develop a general framework for constructing and analyzing overlapping sliding-window statistics for dependence and trend detection. For a fixed window size, the overlapping blocks form a Markov chain, and the asymptotic variance of any centered window statistic is determined by the covariance operator of this chain. Using its spectral structure, we obtain an orthogonal decomposition of the space of window functions into components associated with different overlap levels. This leads to a natural notion of incremental dependence information: the part of a statistic that captures exactly the new information introduced by enlarging the window. We give an explicit procedure for extracting these components and apply the method to two classes of examples. For correlation detection, we study symmetric polynomial window functions and identify their informative projected part. For trend detection, we analyze localized rank-based statistics and isolate the contribution of the largest newly introduced lag. The examples also show that different statistics may exhibit different local detection scales, including nonclassical ones. The same viewpoint leads to a natural quantitative measure of incremental information, which can be used to assess how much new dependence structure is captured as the window size increases, and to guide scale selection. Overall, the paper provides a systematic method for designing overlapping-window tests and deriving their asymptotic normalization and local behavior.

Figures

Figures reproduced from arXiv: 2607.13173 by the authors.

Figure 1
Figure 1. Empirical power of g1 under local AR(1) alternatives. Left: correct scaling ρn = c n−1/4 , showing approximately stable rejection prob￾abilities across n. Right: incorrect scaling ρn = c n−1/2 , showing rejection probabilities drifting toward the nominal level. Interpretation. These results demonstrate that the coefficients α∆ play a fundamental role: they control how signal accumulates across lags. While K is first… view at source ↗
Figure 2
Figure 2. Empirical power of the unprojected g under local AR(1) alter￾natives. Left: correct scaling ρn = c n−1/4 , showing approximately stable rejection probabilities across n. Right: incorrect scaling ρn = c n−1/2 , showing rejection probabilities drifting toward the nominal level. Empirical local behavior. In contrast to the full statistic K, the projected statistic K1 does not exhibit a stable local asymptotic regime un… view at source ↗
Figure 3
Figure 3. Effect of weighting coefficients α∆ on the power of K under local drift alternatives (n = 100). Increasing weights enhance sensitivity, while endpoint-only weighting leads to substantial loss of power. 0.00 0.02 0.04 0.06 200 400 800 1000 1200 n Power c 0.05 0.3 0.5 0.75 βn = cn−1 2 0.00 0.25 0.50 0.75 1.00 100 200 400 600 800 1000 n Power c 0.25 0.5 0.75 1 βn = cn−1 4 0.00 0.25 0.50 0.75 1.00 50 75 100 200 n Power … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Empirical power of K1 under local drift alternatives βn = c n−γ for three candidate scalings. Left: γ = 1/2, where the signal remains weak. Middle: γ = 1/4, where the power continues to increase with n rather than stabilizing. Right: γ = 1/6, where the power increases …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references

  1. [1]

    Alhakim,On the eigenvalues and eigenvectors of an overlapping Markov chain, Probability Theory and Related Fields, 128 (2004), pp

    A. Alhakim,On the eigenvalues and eigenvectors of an overlapping Markov chain, Probability Theory and Related Fields, 128 (2004), pp. 589-605

    2004

  2. [2]

    Alhakim,Hadamard matrices, quaternions, and the Pearson chi-square statistic, Statistical Pa- pers, 65(8), (2024), pp

    A. Alhakim,Hadamard matrices, quaternions, and the Pearson chi-square statistic, Statistical Pa- pers, 65(8), (2024), pp. 5273–5291

    2024

  3. [3]

    Alhakim, J

    A. Alhakim, J. Kawczak, and S. Molchanov,On the class of nilpotent Markov chains, I: the spectrum of covariance operator, Markov Processes and Related Fields 10 (2004), pp. 629-652

    2004

  4. [4]

    Cabilio, Y

    P. Cabilio, Y. Zhang, X. Chen,Bootstrap rank tests for trend in time series, Environmetrics, V 24, 8 (2013), pp. 537 - 549

    2013

  5. [5]

    The serial test for sampling numbers and other tests of randomness

    Good, I.J., (1953) “The serial test for sampling numbers and other tests of randomness”. In Pro- ceedings of Cambridge Philosophical Society, 49, 276-284

    1953

  6. [6]

    M. I. Gordin and B. A. Lifˇ sic,Central limit theorem for stationary Markov processes, Soviet Math- ematics Doklady, 19, (1978), pp. 392–394

    1978

  7. [7]

    Hoeffding and H

    W. Hoeffding and H. Robbins, The central limit theorem for dependent random variables,Duke Mathematical Journal15 (1948), 773–780

    1948

  8. [8]

    Janson, Gaussian Hilbert Spaces, Cambridge University Press, 1997

    S. Janson, Gaussian Hilbert Spaces, Cambridge University Press, 1997

    1997

Show all 21 references
  1. [9]

    M. G. Kendall, Rank Correlation Methods, Griffin, 1955

    1955

  2. [10]

    Le Cam, Asymptotic Methods in Statistical Decision Theory, Springer, 1986

    L. Le Cam, Asymptotic Methods in Statistical Decision Theory, Springer, 1986

    1986

  3. [11]

    Le Cam and G

    L. Le Cam and G. L. Yang, Asymptotics in Statistics: Some Basic Concepts, Springer, 2000

    2000

  4. [12]

    Lehmann and J.P

    E.L. Lehmann and J.P. Romano, Testing Statistical Hypotheses, 3rd. Ed., Springer, 2005

    2005

  5. [13]

    L’Ecuyer, R

    P. L’Ecuyer, R. Simard and S. Wegenkittl,Sparse Serial Tests of Uniformity for Random Number Generators, SIAM Journal On Scientific Computing, 24, 2 (2002), pp. 652-668

    2002

  6. [14]

    Mann,Nonparametric tests against trend, Econometrica, Journal of the Econometric Society, (1945) pp

    H.B. Mann,Nonparametric tests against trend, Econometrica, Journal of the Econometric Society, (1945) pp. 245-259. 25

    1945

  7. [15]

    S. P. Meyn, and R. L. Tweedie, Markov Chains and Stochastic Stability, 2nd. Ed., Cambridge University Press, 2009

    2009

  8. [16]

    Marsaglia,A Current View of Random Number Generators, Computer Science and Statistics, Elsevier Science Publisher B.V., North-Holland, 1985

    G. Marsaglia,A Current View of Random Number Generators, Computer Science and Statistics, Elsevier Science Publisher B.V., North-Holland, 1985

    1985

  9. [17]

    S. M. Pincus,Approximate entropy as a measure of system complexity, Proceedings of the National Academy of Science, USA, 88, (1991), pp. 2297–2301

    1991

  10. [18]

    Pincus and B.H

    S.M. Pincus and B.H. Singer,Randomness and Degrees of Irregularity. Proceedings of the National Academy of Science, USA, 93 (1996), pp. 2083–2088

    1996

  11. [19]

    R. J. Serfling, Approximation Theorems of Mathematical Statistics, Wiley, New York, 1980

    1980

  12. [20]

    Xu and W

    X. Xu and W. Tsang,An Empirical Study on the Power of the Overlapping Serial Test, proc. Asia Simulation Conference 2007, J.W. Park, T.G. Kim and Y. B. Kim, Eds. Seoul, Korea, (2007), pp. 298-306

    2007

  13. [21]

    van der Vaart, Asymptotic Statistics, Cambridge University Press, 1998

    Aad W. van der Vaart, Asymptotic Statistics, Cambridge University Press, 1998. AppendixA.Mean expansion under Gaussian AR(1) We sketch the derivation of the expansion mg1(ρ) =E ρ[g1(Y1)] =ρ 2 +O(ρ 3) for the statisticg 1 defined in (4.8), under a Gaussian AR(1) model with para...

    1998

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.