Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Sequential One-Sided Hypothesis Testing of Markov Chains

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read The paper proves that a sequential test for Markov chains using any estimator with logarithmic regret matches the expected stopping time of the SPRT that already knows the alternative.

desk verdict A genuinely adaptive one-sided SPRT for Markov chains with a regret-based estimator; the proof needs cleanup but the idea and type-1 bound are solid. read the letter →

arxiv 2501.13187 v1 pith:GFIZRRJN submitted 2025-01-22 stat.AP math.STstat.TH

classification stat.APmath.STstat.TH MSC 62L1062M0260J10
keywords sequentialhypothesistestingMarkovchainsSPRTcompositealternativeanomalydetectionregretboundJeffreysmixturemartingale
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks how quickly a stream of observations from a Markov chain can be tested against a known transition matrix $P$ as the null hypothesis, when the alternative is any other ergodic chain. The authors propose a one-sided sequential test: keep sampling as long as the running likelihood ratio stays below $1/\alpha$, and reject $P$ the first time it crosses. Because the alternative is not known, the likelihood ratio is built from a causal estimate of the data-generating chain; the test's stopping time then depends on a regret bound for that estimator. The central result is that any estimator with cumulative log-loss regret $O(\log t)$ makes the expected stopping time under an alternative $Q$ scale as $\log(1/(\alpha D_M(Q\|P)))/D_M(Q\|P)$, matching the sample size of an oracle sequential probability ratio test (SPRT) that knows $Q$. This means anomaly detection and similar monitoring tasks can adapt to the unknown difficulty of the deviation, without knowing the deviation in advance.

What carries the argument

The engine of the argument is the empirical likelihood-ratio process. Instead of an unknown alternative $Q$, the test uses a causal estimator $\hat{Q}_t$ built from the previous samples and updates $L_t = L_{t-1} \hat{Q}_t(X_t|X_{t-1})/P(X_t|X_{t-1})$. Under $H_0$ this quotient has conditional expectation 1, making $L_t$ a non-negative martingale, which immediately controls the false-rejection probability through Ville's inequality. Under $H_1$, the quality of $\hat{Q}_t$ is measured by its worst-case pointwise cumulative log-loss regret $r_t = \max \sum_i (\log(1/\hat{q}_i(X_i)) - \log(1/q(X_i)))$; when $r_t = O(\log t)$, the test statistic is close enough to the oracle statistic $\prod Q(X_i|X_{i-1})/P(X_i|X_{i-1})$ that the stopping time is controlled by a Markov-chain version of Wald's identity. That identity turns $E[\log L^*_\tau]$ into $D_M(Q\|P) E[\tau]$ plus a bounded drift term, producing the SPRT-matching bound. The reachability problem of rarely visited states is handled by estimators such as the Jeffreys mixture over ergodic Markov chains rather than row-by-row add-constant estimates.

What would settle it

For a fixed ergodic alternative $Q$ and null $P$, run Algorithm 1 with a Jeffreys-mixture estimator for decreasing $\alpha$; if the empirical average stopping time grows faster than $\log(1/\alpha)/D_M(Q\|P)$, for instance if $E[\tau] D_M(Q\|P)/\log(1/\alpha)$ diverges, then the claimed SPRT-matching bound is false.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a composite, one-sided sequential test can asymptotically match the simple-hypothesis SPRT. Algorithm 1 forms $L_t = \prod_{i=1}^t \hat{Q}_i(X_i|X_{i-1})/P(X_i|X_{i-1})$ and stops at $\tau = \inf\{t: L_t \ge 1/\alpha\}$, declaring $H_1$. Under $H_0$, $L_t$ is a non-negative martingale starting at 1, so Ville's inequality gives type-1 error at most $\alpha$ for every valid estimator. If the estimator's worst-case pointwise cumulative log-loss regret is $O(\log t)$, then for an ergodic alternative $Q$ whose likelihood ratios $Q(j|i)/P(j|i)$ are bounded by a finite constant, $E[\tau] = O(\log(1/(\alpha D_M(Q\|P)))/D_M(Q\|P))$, where $D_M$ is the stationary-averaged KL divergence. The bound matches the expected sample size of the SPRT that knows $Q$, so the test adapts to the hardness of the alternative with no prior information.

Load-bearing premise

The load-bearing premise is that the deployed estimator has worst-case cumulative log-loss regret $O(\log t)$ uniformly over sample paths and chain parameters, and that the alternative chain is ergodic with finite likelihood ratios $Q(j|i)/P(j|i)$; if any of these fails, the proof that the test keeps pace with the known-alternative SPRT no longer goes through.

Editorial extensions

If this is right

  • Any valid causal estimator gives type-1 error at most $\alpha$, so the false-rejection guarantee is robust to the estimator choice.
  • With an $O(\log t)$ regret estimator, the expected stopping time under an unknown alternative matches, up to constants, the SPRT that is told the alternative, so no prior knowledge of the deviation is needed.
  • The test is one-sided: under the null it is allowed to run indefinitely, which is the desired behavior for continuous monitoring and anomaly detection.
  • The bound is expressed through the stationary-averaged KL divergence $D_M(Q\|P)$, so harder-to-detect alternatives automatically require more samples, while easy alternatives stop quickly.
  • Known estimators with the required regret guarantee exist in both the i.i.d. case and the Markov case, so the theorem is not vacuous.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that replacing the $O(\log t)$ regret condition with a super-logarithmic one would produce a correspondingly slower expected stopping time, so the estimator's regret is the natural tuning knob for the speed-robustness trade-off in practice.
  • We infer that the same statistic can be applied when the null itself is estimated from a finite warm-up sample, but then $\alpha$ should be reinterpreted as conditional on that estimate; a formal treatment of this two-stage setting is not in the paper.
  • We infer a concrete testable extension: run the algorithm with a Bayesian context-tree estimator on data with variable-length memory; if such an estimator has $O(\log t)$ regret, the SPRT-matching bound should transfer directly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies one-sided sequential hypothesis testing for finite-state Markov chains: given a known null transition matrix P, decide whether a stream X1, X2, ... is generated by P or by some unknown alternative Q. The proposed Algorithm 1 constructs an empirical likelihood ratio L_t using a causal estimator \hat{Q}_t and stops when L_t ≥ 1/α. The type-1 error bound follows from the nonnegative martingale property of L_t under H0 and Ville's inequality. The central performance claim (Theorem 2) is that, whenever the estimator has worst-case regret r_t = O(log t), the expected stopping time under an ergodic alternative Q satisfies E[τ] = O( log(1/(α D_M(Q||P))) / D_M(Q||P) ), asymptotically matching the SPRT that knows Q. The paper discusses the Jeffreys mixture estimator of Takeuchi and Barron for Markov chains and reports simulations comparing the sequential test with the fixed-length test of Wolfer and Kontorovich.

Significance. The proposed test is conceptually appealing: it decouples the stopping rule from the choice of estimator and would give a fully adaptive, composite one-sided test with oracle-SPRT performance. The type-1 error analysis via Ville's inequality is clean and does not depend on the estimator beyond producing valid conditional distributions. The paper also provides a useful discussion of why the Markov setting is harder than the i.i.d. case and an interesting comparison with fixed-length identity testing. If the expected-stopping-time result can be made rigorous, the paper would be a valuable contribution to sequential analysis and anomaly detection. However, the current manuscript does not yet supply a complete proof of the headline bound, because the Markov regret assumption is not formally defined and the key pathwise domination step is not justified.

major comments (4)
  1. [Definition 2, §III-B and §III-C] Definition 2 defines the cumulative pointwise worst-case regret only for i.i.d. distributions q ∈ Δ^{m−1}. Theorem 2, however, invokes an analogous regret guarantee for Markov estimators {\hat{Q}_t} without formally defining the Markov version or stating over which class of alternatives the supremum is taken. Without a deterministic, pathwise Markov regret bound—i.e., a pointwise inequality of the form log L*_t − log L_t ≤ r_t holding for every sample path and every t—the inequality τ ≤ τ* in §III-C does not follow from the stated assumptions. The paper needs to define the Markov regret precisely and either prove the O(log t) pointwise bound for the Jeffreys mixture estimator or cite a theorem in [9] that supplies it.
  2. [§III-C, Eq. (4)] The displayed derivation leading to Eq. (4) contains a non-sequitur that is load-bearing for the expected stopping time bound. The proof defines τ* via an oracle statistic L*_t, then writes "E[log L_{τ*}] ≤ log 1/α + E[log τ*] + C". This mixes L and L*: the oracle stopping time controls log L*_{τ*}, not log L_{τ*}. Furthermore, the overshoot term is written as E[log L*_τ − log L_{τ*−1}], which should be E[log L*_{τ*} − log L*_{τ*−1}]. The intended inequality is likely E[log L*_{τ*}] ≤ log(1/α) + E[r_{τ*}] + O(1), with r_t = O(log t) converted to E[log τ*] via Jensen; this needs to be written out correctly, and the uniform bound on the overshoot needs a proof rather than an assertion.
  3. [Remark 4, §III-C] Remark 4 states that the bounded-likelihood assumption max_{i,j} Q(j|i)/P(j|i) ≤ c' < ∞ can be removed with "a slightly more involved argument", but no proof or reference is given. Since the stated Theorem 2 uses c' to bound the per-step log-likelihood increments and to apply the Markov Wald identity, the advertised claim in the Introduction that no assumption is made on the alternative is not supported by the theorem as written. The authors should either prove the c' < ∞ case rigorously with a finite overshoot argument, or temper the claims in the abstract and introduction to reflect the condition actually used.
  4. [§III-D, Jeffreys mixture claim] The paper asserts that the Jeffreys mixture estimator for Markov chains attains regret O((m−1)m/2 log(t/2π)) and cites [9], but does not specify whether the cited result is pointwise (pathwise) or in expectation and does not give a theorem number. This matters because the proof of Theorem 2 requires a pointwise deterministic regret bound for every sample path, not merely an averaged or high-probability bound. The authors should state the precise regularity and uniformity of the regret bound obtained from [9] or prove the needed pathwise bound; otherwise the substitution of r_t in the oracle-stopping-time argument is unjustified.
minor comments (5)
  1. [§III-A] There is a typo in "Our alogorithm and test work with any method..." — "alogorithm" should be "algorithm".
  2. [§III-C, proof of Theorem 2] In the definition of τ*, the right-hand side is written as a set ("≤ {t ≥ 1 : ... } =: τ*"), but τ* should be defined as the infimum of that set, i.e., τ* := inf{t ≥ 1 : log L*_t ≥ log(1/α) + r_t}.
  3. [§III-C, martingale check] In the display verifying the martingale property, the summation index is misprinted as "∑_{i=j}"; it should be "∑_{j∈[m]}".
  4. [Eq. (8) and Fig. 2] The caption of Fig. 2 refers to "Problem 8", which is likely Eq. (8); please make the cross-reference explicit and consistent.
  5. [Definition 2] The notation "r_t((\hat{q}_i)^t_{i=1})" is awkward: the subscript t on the left is redundant, and the definition as a maximum over trajectories and a supremum over q is not clearly separated. A cleaner notation would help the reader distinguish the pointwise regret from its worst-case bound.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the main theorem is a conditional reduction from estimator regret to SPRT-matching stopping time, with independent external ingredients.

full rationale

The paper's central claim is Theorem 2: if a causal estimator has worst-case pointwise regret r_t = O(log t), then the proposed one-sided SPRT-type test has type-1 error at most alpha and expected stopping time O(log(1/(alpha D_M(Q||P)))/D_M(Q||P)). The type-1 bound follows from Ville's inequality applied to the nonnegative martingale L_t, and the expected-stopping-time bound follows by comparing L_t with the oracle likelihood ratio L*_t at a cost of the estimator's regret, then applying a Markov-chain Wald identity from Moustakides to evaluate E[log L*_{tau*}]. Neither step assumes the conclusion: the regret bound is an explicit input condition, and the SPRT-matching constant D_M(Q||P) enters through the independent Wald identity rather than through the algorithm's construction. The existence of estimators with the required regret is cited to Takeuchi, Kawabata, and Barron, with no author overlap with the present paper, so there is no self-citation chain. No parameter is fitted and later reported as a prediction, and no quantity is defined in terms of the stopping time it is supposed to explain. The manuscript does contain rigor concerns that are not circularity: Definition 2 states pointwise worst-case regret in the i.i.d. setting and the Markov analogue is only asserted via citation, and Remark 4 claims the c' < infinity likelihood-ratio assumption can be removed without proof. These affect proof completeness, not whether the derivation reduces to its own inputs. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities and fits no free parameters to data. Its results depend on standard statistical tools (Ville, Wald-Markov identity), a known estimator from the literature, and explicit regularity assumptions on the alternative chain.

assumptions (5)
  • standard math Ville's inequality for nonnegative supermartingales.
    Used to bound P_H0(tau < infinity) <= alpha; stated as Theorem 3 in Appendix A-A.
  • domain assumption Wald's identity for Markov chains (Moustakides 1999).
    Used in Theorem 2 to express E[log L*_{tau*}] as D_M(Q||P) E[tau*] plus a bounded drift term; conditions on the eigenvalues of Q are summarized in Appendix A-B.
  • domain assumption Regret bound O(log t) for the chosen estimator, instantiated by Takeuchi and Barron's Jeffreys mixture.
    Theorem 2 assumes an estimator with worst-case regret r_t = O(log t); the paper cites [9] for the Jeffreys mixture achieving r_t = O((m-1)m/2 log(t/2pi)).
  • domain assumption The alternative chain Q is ergodic and satisfies max_{i,j} Q(j|i)/P(j|i) <= c' for a finite constant c'.
    Explicit assumption in Theorem 2; bounds the overshoot of the log-likelihood ratio and supports the Wald identity drift bound.
  • domain assumption The estimator Q-hat_t is a valid probability distribution adapted to the observed history.
    Needed for the (super)martingale property under H0 and for the regret definition; stated in Section III-A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sequential One-Sided Hypothesis Testing of Markov Chains." pith.science (2026). https://pith.science/paper/GFIZRRJN

@misc{pith2026250113187,
  author       = {Pith},
  title        = {Pith review of: Sequential One-Sided Hypothesis Testing of Markov Chains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GFIZRRJN}},
  note         = {Machine review of arXiv:2501.13187}
}
abstract

We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory of a Markov chain with a known transition matrix $P$ (null hypothesis) or not (composite alternative hypothesis). This problem naturally arises in many engineering problems. The main technical challenge is to develop a sequential testing scheme that adapts its sample size to the unknown alternative. Indeed, if we knew the alternative distribution (that is, the transition matrix) $Q$, a natural approach would be to use a generalization of Wald's sequential probability ratio test (SPRT). Building on this intuition, we propose and analyze a family of one-sided SPRT-type tests for our problem that use a data-driven estimator $\hat{Q}$. In particular, we show that if the deployed estimator admits a worst-case regret guarantee scaling as $\mathcal{O}\left( \log{t} \right)$, then the performance of our test asymptotically matches that of SPRT in the simple hypothesis testing case. In other words, our test automatically adapts to the unknown hardness of the problem, without any prior information. We end with a discussion of known Markov chain estimators with $\mathcal{O}\left( \log{t} \right)$ regret.

Figures

Figures reproduced from arXiv: 2501.13187 by the authors.

Figure 1
Figure 1. Power of the fixed length test of Wolfer and Kontorovic [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Average stopping time of our algorithm instantiated [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Asymptotically Optimal Sequential Testing with Markovian Data

    math.ST 2026-02 conditional novelty 7.0 of 10

    For ergodic finite-state Markov chains, the optimal expected stopping time of an α-correct one-sided sequential test is asymptotically log(1/α)/Dinf_M(Q,P), where Dinf is the least stationary-weighted KL divergence fr...

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [9]

    Properties o f Jeffreys mixture for Markov sources,

    J. Takeuchi, T. Kawabata, and A. R. Barron, “Properties o f Jeffreys mixture for Markov sources,” IEEE transactions on information theory , vol. 59, no. 1, pp. 438–457, 2012

  2. [1]

    Large deviations, hypotheses testing, a nd source coding for finite Markov chains,

    S. Natarajan, “Large deviations, hypotheses testing, a nd source coding for finite Markov chains,” IEEE Trans. Inf. Theory, vol. 31, pp. 360–365, 1985. [Online]. Available: https://api.semanticscholar.org/CorpusID:27423979

  3. [2]

    Sequential Analysis of Dependent Obse rvations. I,

    R. M. Phatarfod, “Sequential Analysis of Dependent Obse rvations. I,” Biometrika, vol. 52, no. 1/2, pp. 157–165, 1965. [Online]. Available: http://www.jstor.org/stable/2333821

  4. [3]

    Sequential Tests of Statistical Hypotheses,

    A. Wald, “Sequential Tests of Statistical Hypotheses,” The Annals of Mathematical Statistics , vol. 16, no. 2, pp. 117 – 186, 1945. [Online]. Available: https://doi.org/10.1214/aoms/1177731118

  5. [4]

    Minimax optimal se quential hypothesis tests for Markov processes,

    M. Fauss, A. M. Zoubir, and H. V . Poor, “Minimax optimal se quential hypothesis tests for Markov processes,” The Annals of Statistics , vol. 48, no. 5, pp. 2599–2621, 2020

  6. [5]

    Asymptotically optimal tests for multin omial distribu- tions,

    W. Hoeffding, “Asymptotically optimal tests for multin omial distribu- tions,” The Annals of Mathematical Statistics , pp. 369–401, 1965

  7. [6]

    Generalized sequential proba bility ratio test for separate families of hypotheses,

    X. Li, J. Liu, and Z. Ying, “Generalized sequential proba bility ratio test for separate families of hypotheses,” Sequential analysis, vol. 33, no. 4, pp. 539–563, 2014

  8. [7]

    Statistical anomaly det ection via composite hypothesis testing for markov models,

    J. Zhang and I. C. Paschalidis, “Statistical anomaly det ection via composite hypothesis testing for markov models,” IEEE Transactions on Signal Processing , vol. 66, no. 3, pp. 589–602, 2018

Show all 22 references
  1. [8]

    Sequential algorithms for testing identity and closeness of distributions,

    O. Fawzi, N. Flammarion, A. Garivier, and A. Oufkir, “Sequential algorithms for testing identity and closeness of distributions,” ArXiv, vol. abs/2205.06069, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:248721783

  2. [10]

    Minimax Testing of Ident ity to a Reference Ergodic Markov Chain,

    G. Wolfer and A. Kontorovich, “Minimax Testing of Ident ity to a Reference Ergodic Markov Chain,” in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , ser. Proceedings of Machine Learning Research, S. Chiappa a nd R. Calandra, ...

  3. [11]

    A survey on distribution testing: Y our d ata is big. but is it blue?

    C. L. Canonne, “A survey on distribution testing: Y our d ata is big. but is it blue?” Theory of Computing , pp. 1–100, 2020

  4. [12]

    Exact goodness-of-fit tests for markov chains,

    J. Besag and D. Mondal, “Exact goodness-of-fit tests for markov chains,” Biometrics, vol. 69, no. 2, pp. 488–496, 2013

  5. [13]

    Testing for the markov property in t ime series,

    B. Chen and Y . Hong, “Testing for the markov property in t ime series,” Econometric Theory , vol. 28, no. 1, pp. 130–178, 2012

  6. [14]

    Extension of Wald’s first lemma to Ma rkov pro- cesses,

    G. V . Moustakides, “Extension of Wald’s first lemma to Ma rkov pro- cesses,” Journal of applied probability , vol. 36, no. 1, pp. 48–59, 1999

  7. [15]

    On Learning Distributions from their Samples,

    S. Kamath, A. Orlitsky, D. Pichapati, and A. T. Suresh, “ On Learning Distributions from their Samples,” in Proceedings of The 28th Conference on Learning Theory , ser. Proceedings of Machine Learning Research, P . Gr¨ unwald, E. Hazan, and S. Kale, Eds., vol. 40. Paris, Franc...

  8. [16]

    The performance of unive rsal encod- ing,

    R. Krichevsky and V . Trofimov, “The performance of unive rsal encod- ing,” IEEE Transactions on Information Theory , vol. 27, no. 2, pp. 199– 207, 1981

  9. [17]

    An automatic inequality pro ver and instance optimal identity testing,

    G. V aliant and P . V aliant, “An automatic inequality pro ver and instance optimal identity testing,” SIAM Journal on Computing , vol. 46, no. 1, pp. 429–455, 2017

  10. [18]

    Bayesian context trees: Modelling and exa ct inference for discrete time series,

    I. Kontoyiannis, L. Mertzanis, A. Panotopoulou, I. Pap ageorgiou, and M. Skoularidou, “Bayesian context trees: Modelling and exa ct inference for discrete time series,” Journal of the Royal Statistical Society Series B: Statistical Methodology , vol. 84, no. 4, pp. 1287–1323, 2022

  11. [19]

    DeepAR: Probabilistic forecasting with autoregressive recurrent networks,

    D. Salinas, V . Flunkert, J. Gasthaus, and T. Januschows ki, “DeepAR: Probabilistic forecasting with autoregressive recurrent networks,” Inter- national journal of forecasting , vol. 36, no. 3, pp. 1181–1191, 2020

  12. [20]

    A survey of methods fo r time series change point detection,

    S. Aminikhanghahi and D. J. Cook, “A survey of methods fo r time series change point detection,” Knowledge and information systems , vol. 51, no. 2, pp. 339–367, 2017

  13. [21]

    Chang e-point detection and segmentation of discrete data using bayesian context tr ees,

    V . Lungu, I. Papageorgiou, and I. Kontoyiannis, “Chang e-point detection and segmentation of discrete data using bayesian context tr ees,” arXiv preprint arXiv:2203.04341, 2022

  14. [22]

    Asymptotic operating characteristics of a n optimal change point detection in hidden markov models,

    C.-D. Fuh, “Asymptotic operating characteristics of a n optimal change point detection in hidden markov models,” Annals of Statistics , pp. 2305–2339, 2004

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.