Pith. sign in

REVIEW 2 major objections 4 minor 17 references

Runs and Bootstrap Tests For Signal Feature Significance

T0 review · 2 major / 4 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Markov recursion plus permutation and bootstrap height tests assign non-parametric significance to peaks and flats in signals, including multi-modality detected via spacing.

desk verdict Clean multi-symbol longest-run recursion plus usable height tests for peaks and flats; solid methods note with one conventional exchangeability caveat. read the letter →

arxiv 2607.09913 v1 pith:EBUJJIIZ submitted 2026-07-10 stat.ME

Greg Kreider This is my paper
classification stat.ME MSC 62G1060J1062M02
keywords signalfeaturesignificancerunstestspacingMarkovchainlongestrunbootstrappermutationmulti-modality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper supplies exact and sampling-based tests that decide whether a peak or flat observed in data is unusually tall or unusually level, without assuming a parametric noise model. It first gives a recursive formula, obtained from a Markov chain on run lengths, for the probability of the longest run of any symbol. It then builds a permutation test on the signed runs inside a candidate feature and a bootstrap test on the raw step differences, both of which return a height quantile. These procedures apply directly to spacing series, so that local peaks and flats in the spacings can be used to locate modes and anti-modes. The result is a practical, distribution-free toolkit for feature significance that works on both discrete and continuous data.

What carries the argument

The run-transition matrix R (block form with diagonal blocks A that continue a run and off-diagonal blocks B that reset it) whose powers give the absorption probability at length L+1; the subsequent height quantiles are obtained by permuting the observed runs (or bootstrapping the observed steps) inside the feature.

What would settle it

Generate many synthetic series whose known transition matrix differs markedly inside versus outside a planted feature; if the reported height quantiles still cluster near the nominal significance level, the exchangeability claim fails.

Watch

Extended reading notes

Core claim

The probability that the longest run is at most L after N steps equals the weighted top-right block of the powered run-transition matrix R^N; the same matrix construction yields the full distribution of run length. From that foundation the paper constructs a permutation test of reconstructed feature height and a bootstrap test of raw-step height, both of which correctly flag significant peaks and flats.

Load-bearing premise

The global transition rates (or the pool of all step sizes after discarding extreme tails) may be used as a null generator for every local feature, treating the steps inside a candidate peak or flat as exchangeable with the rest of the series.

Editorial extensions

If this is right

  • Any quantized or continuous series can be scanned for statistically supported peaks and flats without fitting a noise model.
  • Spacing-based multi-modality detection becomes a fully non-parametric procedure whose local features carry explicit p-values.
  • Flats as well as peaks receive quantitative significance, allowing tests of local constancy or scale changes.
  • The longest-run recursion supplies an exact null for run-based process-control rules that previously relied on coarse combinatorial approximations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same bootstrap construction can be applied to any additive path functional (area, roughness, number of crossings) simply by replacing the height extractor.
  • When the series is known to be a filtered process, the empirical transition matrix already incorporates the filter-induced dependence, so the tests remain valid without further whitening.
  • The method immediately yields a non-parametric competitor to classical changepoint or mode-hunting procedures that rely on kernel density estimates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper derives a recursive expression for the longest-run distribution in multi-symbol sequences by constructing an absorbing Markov chain R from the empirical transition matrix T (eqs. 1–7), verified by Monte-Carlo in Fig. 1. It then presents a permutation test on signed run lengths that form a feature (Listing 1) and a bootstrap test on raw step differences (Listing 2) to obtain height quantiles for both peaks and flats. These are applied to interval spacing series to locate multi-modality, illustrated on soil organic-matter data and implemented in the Dimodal R package.

Significance. The Markov recursion supplies an exact, matrix-power expression for P(longest run ≤ L) under a general transition matrix, cleanly extending classical combinatorial runs results and matching simulation. The two resampling procedures are correctly specified standard Monte-Carlo tests that correctly handle both large peak heights and small flat heights without parametric noise models. The Dimodal package supplies a reproducible reference implementation. Together these tools give a practical non-parametric route to feature significance in spacing-based multi-modality analysis; the main practical limitation is the conventional exchangeability assumption between global and local step statistics, which future type-I/power studies can quantify.

major comments (2)
  1. [I.A, II.B, III] Sections I.A and II.B (and the Dimodal description in III): the longest-run probabilities and both height tests treat the global empirical transition matrix T (or the global pool of step differences after 3-sd tail trimming) as a valid null generator for every local feature. This exchangeability assumption is conventional for non-parametric runs/bootstrap tests but is load-bearing for the multi-modality claims; controlled type-I and power simulations under known multi-modal densities are needed to quantify robustness.
  2. [II.B, III] II.B and III: free parameters (90 % peak-width fraction, 10 % ripple, 3-sd tail exclusion, N_perm/N_boot) are introduced without sensitivity analysis. Their influence on the soil-data conclusions (Fig. 4, Tables 2–3) should be reported so that readers can judge stability of the reported p-values.
minor comments (4)
  1. [II.A–B] Listing 1 step 5 and Listing 2 step 4: the half-count continuity correction for discrete quantiles is ad-hoc; a brief justification or comparison with mid-p / randomization alternatives would improve transparency.
  2. [I.A] Equation (1) and surrounding text: block sizes of the (L+1)m imes (L+1)m matrix R are stated but the layout of the identity absorbing block could be made more explicit for readers implementing the recursion.
  3. [I.A] Fig. 1 caption and text: clarify whether the grey band is a pointwise or simultaneous 95 % interval over the 2500 trials.
  4. [References] References: the classical runs literature is well covered; a short pointer to more recent Markov-chain runs work (post-Fu & Koutras) would help situate the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: longest-run distribution is ordinary absorbing-chain powering; height tests are pure Monte-Carlo under the empirical null.

full rationale

The paper’s central results are self-contained. Equations (1)–(7) construct an absorbing Markov chain R from the empirical transition matrix T = A + B and obtain P(longest run ≤ L) by ordinary matrix powering (or the equivalent recursion (4)); Fig. 1 confirms the calculation against independent simulation. Listings 1 and 2 are standard permutation and bootstrap procedures that re-sample the observed runs or step differences and count the fraction of reconstructed heights that exceed the observed feature height; no parameter is fitted and later re-used as a “prediction.” The sole self-citation is to the companion Dimodal package that implements the same algorithms; it is not load-bearing for any uniqueness claim or uniqueness theorem. The exchangeability assumption that the global empirical null is valid for every local feature is a conventional modeling choice, not a circular reduction. Consequently the derivation chain contains no self-definitional step, no fitted-input-called-prediction, and no self-citation that forces the result.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper rests on standard Markov-chain theory, the usual exchangeability assumptions of permutation and bootstrap tests, and a handful of conventional tuning constants (sample sizes, ripple percentages, tail cut-offs). No new physical entities are postulated; free parameters are algorithmic choices rather than fitted scientific constants.

free parameters (5)
  • N_perm = 5000
    Number of random permutations used to estimate the height quantile; set to 5000 for ‘stable results’ without further justification.
  • N_boot
    Number of bootstrap replicates for the continuous-height test; left unspecified beyond ‘repeat N_boot times’.
  • ripple_fraction = 0.10
    Maximum relative height allowed inside a ‘flat’ feature; fixed at 10 % by convention.
  • peak_width_fraction = 0.90
    Fraction of peak height used to define feature extent (FWHM-like); fixed at 90 %.
  • tail_exclusion_sd = 3
    Number of standard deviations beyond which spacing values are dropped from the bootstrap pool; set to 3.
assumptions (3)
  • domain assumption Successive quantized symbols form a time-homogeneous Markov chain of finite order whose transition matrix can be estimated by simple counting.
    Invoked at the start of Section I.A to justify construction of T and R; standard for run statistics but not verified for every data type.
  • domain assumption Under the null, the ordered run lengths (or raw steps) inside a candidate feature are exchangeable with those drawn from the global empirical distribution, subject only to the no-adjacent-same-symbol constraint.
    Underpins both Listing 1 and Listing 2; classical permutation/bootstrap premise.
  • standard math Matrix powering of the block-structured run-transition matrix R yields the exact absorption probabilities for the longest run.
    Ordinary Chapman–Kolmogorov; used to obtain eqs. (2)–(7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Runs and Bootstrap Tests For Signal Feature Significance." pith.science (2026). https://pith.science/paper/EBUJJIIZ

@misc{pith2026260709913,
  author       = {Pith},
  title        = {Pith review of: Runs and Bootstrap Tests For Signal Feature Significance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBUJJIIZ}},
  note         = {Machine review of arXiv:2607.09913}
}
read the original abstract

Runs tests have long been used as a non-parametric check if data contains a non-random signal. We derive a recursive expression for the distribution of the longest run using Markov chain theory. Next we develop a permutation test on the runs comprising a feature to get the probability of its height. This leads finally to a bootstrap test on the height using the raw, continuous data. Such a test can evaluate not only the large heights of peaks but also the small heights of flats. We can apply these tests to features in the spacing of data to detect and locate multi-modality.

Figures

Figures reproduced from arXiv: 2607.09913 by the authors.

Figure 1
Figure 1. Verification of the longest run distribution. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Raw signal and simplified cumulative signed difference (c.s.d.). [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Two signals, the top with large differences and the bottom not. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Spacing analysis of organic matter density near the Maas river. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references

  1. [1]

    C. W. Helstrom,Elements of Signal Detection and Estimation. Engle- wood Cliffs, New Jersey: PTR Prentice Hall, 1995

    1995

  2. [2]

    The distribution of Brownian motion on linear stopping boundaries,

    W. J. Hall, “The distribution of Brownian motion on linear stopping boundaries,”Sequential Analysis, vol. 16, no. 4, pp. 345–352, 1997

    1997

  3. [3]

    Excursions in Brownian motion,

    K. L. Chung, “Excursions in Brownian motion,”Arkiv f ¨or Matematik, vol. 14, no. 1–2, pp. 155–177, Dec. 1976

    1976

  4. [4]

    Some results on the Brownian meander with drift,

    F. Iafrate and E. Orsingher, “Some results on the Brownian meander with drift,”Journal of Theoretical Probability, vol. 33, pp. 1034–1060, 2019

    2019

  5. [5]

    A significance test for time series and other ordered observations,

    W. A. Wallis and G. H. Moore, “A significance test for time series and other ordered observations,” National Bureau of Economic Research, New York, Tech. Rep., Sep. 1941. 5 0 50 100 150 0 1 2 3 4 Interval Spacing index spacing, interval = 15 spacing (dots) scaled by 4 Fig. 4. Spacing analysis of organic matter density near the Maas river

    1941

  6. [6]

    On the theory of runs with some applications to quality control,

    J. Wolfowitz, “On the theory of runs with some applications to quality control,”The Annals of Mathematical Statistics, vol. 14, no. 3, pp. 280– 288, Sep. 1943

    1943

  7. [7]

    The distribution theory of runs,

    A. M. Mood, “The distribution theory of runs,”The Annals of Mathe- matical Statistics, vol. 11, pp. 367–392, 1940

    1940

  8. [8]

    Multiple runs distributions: Recurrences and critical values,

    P. W. Shaughnessy, “Multiple runs distributions: Recurrences and critical values,”Journal of the American Statistical Association, vol. 76, no. 375, pp. 732–736, Sep. 1981

    1981

Show all 17 references
  1. [9]

    Multiple matching and runs by the symbolic method,

    I. Kaplansky and J. Riordan, “Multiple matching and runs by the symbolic method,”The Annals of Mathematical Statistics, vol. 16, pp. 272–277, 1945

    1945

  2. [10]

    On a test whether two samples are from the same population,

    A. Wald and J. Wolfowitz, “On a test whether two samples are from the same population,”The Annals of Mathematical Statistics, vol. 11, pp. 147–162, 1940

    1940

  3. [11]

    Successes, runs, and longest runs,

    A. N. Philippou and F. S. Makri, “Successes, runs, and longest runs,” Statistics and Probability Letters, vol. 4, pp. 211–215, Jun. 1986

    1986

  4. [12]

    Statistical inference about Markov chains,

    T. W. Anderson and L. A. Goodman, “Statistical inference about Markov chains,”The Annals of Mathematical Statistics, vol. 28, no. 1, pp. 89– 110, Mar. 1957

    1957

  5. [13]

    Simulation results for Markov model selection: AIC, BIC and EDC,

    C. C. Y . Dorea, C. R. Goncalves, and P. A. A. Resende, “Simulation results for Markov model selection: AIC, BIC and EDC,” inProceedings of the World Congress on Engineering and Computer Science, vol. II. San Francisco: WCECS, Oct. 2014

    2014

  6. [14]

    Distribution theory of runs: A Markov chain approach,

    J. C. Fu and M. V . Koutras, “Distribution theory of runs: A Markov chain approach,”Journal of the American Statistical Association, vol. 89, no. 427, pp. 1050–1058, Sep. 1994

    1994

  7. [15]

    Asymptotic properties of sample quantiles of discrete distributions,

    Y . Ma, M. G. Genton, and E. Parzen, “Asymptotic properties of sample quantiles of discrete distributions,”Annals of the Institute of Statistical Mathematics, vol. 63, pp. 227–243, 2011

    2011

  8. [16]

    Modality analysis via spacing with Dimodal in R,

    G. Kreider, “Modality analysis via spacing with Dimodal in R,” 2025, in submission

    2025

  9. [17]

    M. G. J. Rikken and R. P. G. van Rijn, “Soil pollution with heavy metals — an inquiry into spatial variation, cost of mapping and the risk evaluation of copper, cadmium, lead and zinc in the floodplains of the Meuse west of Stein, the Netherlands,” Utrecht University, Departme...

    1993

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.