REVIEW 4 major objections 5 minor 1 cited by
Sequential One-Sided Hypothesis Testing of Markov Chains
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper proves that a sequential test for Markov chains using any estimator with logarithmic regret matches the expected stopping time of the SPRT that already knows the alternative.
desk verdict A genuinely adaptive one-sided SPRT for Markov chains with a regret-based estimator; the proof needs cleanup but the idea and type-1 bound are solid. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of the argument is the empirical likelihood-ratio process. Instead of an unknown alternative $Q$, the test uses a causal estimator $\hat{Q}_t$ built from the previous samples and updates $L_t = L_{t-1} \hat{Q}_t(X_t|X_{t-1})/P(X_t|X_{t-1})$. Under $H_0$ this quotient has conditional expectation 1, making $L_t$ a non-negative martingale, which immediately controls the false-rejection probability through Ville's inequality. Under $H_1$, the quality of $\hat{Q}_t$ is measured by its worst-case pointwise cumulative log-loss regret $r_t = \max \sum_i (\log(1/\hat{q}_i(X_i)) - \log(1/q(X_i)))$; when $r_t = O(\log t)$, the test statistic is close enough to the oracle statistic $\prod Q(X_i|X_{i-1})/P(X_i|X_{i-1})$ that the stopping time is controlled by a Markov-chain version of Wald's identity. That identity turns $E[\log L^*_\tau]$ into $D_M(Q\|P) E[\tau]$ plus a bounded drift term, producing the SPRT-matching bound. The reachability problem of rarely visited states is handled by estimators such as the Jeffreys mixture over ergodic Markov chains rather than row-by-row add-constant estimates.
What would settle it
For a fixed ergodic alternative $Q$ and null $P$, run Algorithm 1 with a Jeffreys-mixture estimator for decreasing $\alpha$; if the empirical average stopping time grows faster than $\log(1/\alpha)/D_M(Q\|P)$, for instance if $E[\tau] D_M(Q\|P)/\log(1/\alpha)$ diverges, then the claimed SPRT-matching bound is false.
Extended reading notes
Core claim
On its own terms, the paper establishes that a composite, one-sided sequential test can asymptotically match the simple-hypothesis SPRT. Algorithm 1 forms $L_t = \prod_{i=1}^t \hat{Q}_i(X_i|X_{i-1})/P(X_i|X_{i-1})$ and stops at $\tau = \inf\{t: L_t \ge 1/\alpha\}$, declaring $H_1$. Under $H_0$, $L_t$ is a non-negative martingale starting at 1, so Ville's inequality gives type-1 error at most $\alpha$ for every valid estimator. If the estimator's worst-case pointwise cumulative log-loss regret is $O(\log t)$, then for an ergodic alternative $Q$ whose likelihood ratios $Q(j|i)/P(j|i)$ are bounded by a finite constant, $E[\tau] = O(\log(1/(\alpha D_M(Q\|P)))/D_M(Q\|P))$, where $D_M$ is the stationary-averaged KL divergence. The bound matches the expected sample size of the SPRT that knows $Q$, so the test adapts to the hardness of the alternative with no prior information.
Load-bearing premise
The load-bearing premise is that the deployed estimator has worst-case cumulative log-loss regret $O(\log t)$ uniformly over sample paths and chain parameters, and that the alternative chain is ergodic with finite likelihood ratios $Q(j|i)/P(j|i)$; if any of these fails, the proof that the test keeps pace with the known-alternative SPRT no longer goes through.
Editorial extensions
If this is right
- Any valid causal estimator gives type-1 error at most $\alpha$, so the false-rejection guarantee is robust to the estimator choice.
- With an $O(\log t)$ regret estimator, the expected stopping time under an unknown alternative matches, up to constants, the SPRT that is told the alternative, so no prior knowledge of the deviation is needed.
- The test is one-sided: under the null it is allowed to run indefinitely, which is the desired behavior for continuous monitoring and anomaly detection.
- The bound is expressed through the stationary-averaged KL divergence $D_M(Q\|P)$, so harder-to-detect alternatives automatically require more samples, while easy alternatives stop quickly.
- Known estimators with the required regret guarantee exist in both the i.i.d. case and the Markov case, so the theorem is not vacuous.
Reading between the lines
- We infer that replacing the $O(\log t)$ regret condition with a super-logarithmic one would produce a correspondingly slower expected stopping time, so the estimator's regret is the natural tuning knob for the speed-robustness trade-off in practice.
- We infer that the same statistic can be applied when the null itself is estimated from a finite warm-up sample, but then $\alpha$ should be reinterpreted as conditional on that estimate; a formal treatment of this two-stage setting is not in the paper.
- We infer a concrete testable extension: run the algorithm with a Bayesian context-tree estimator on data with variable-length memory; if such an estimator has $O(\log t)$ regret, the SPRT-matching bound should transfer directly.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies one-sided sequential hypothesis testing for finite-state Markov chains: given a known null transition matrix P, decide whether a stream X1, X2, ... is generated by P or by some unknown alternative Q. The proposed Algorithm 1 constructs an empirical likelihood ratio L_t using a causal estimator \hat{Q}_t and stops when L_t ≥ 1/α. The type-1 error bound follows from the nonnegative martingale property of L_t under H0 and Ville's inequality. The central performance claim (Theorem 2) is that, whenever the estimator has worst-case regret r_t = O(log t), the expected stopping time under an ergodic alternative Q satisfies E[τ] = O( log(1/(α D_M(Q||P))) / D_M(Q||P) ), asymptotically matching the SPRT that knows Q. The paper discusses the Jeffreys mixture estimator of Takeuchi and Barron for Markov chains and reports simulations comparing the sequential test with the fixed-length test of Wolfer and Kontorovich.
Significance. The proposed test is conceptually appealing: it decouples the stopping rule from the choice of estimator and would give a fully adaptive, composite one-sided test with oracle-SPRT performance. The type-1 error analysis via Ville's inequality is clean and does not depend on the estimator beyond producing valid conditional distributions. The paper also provides a useful discussion of why the Markov setting is harder than the i.i.d. case and an interesting comparison with fixed-length identity testing. If the expected-stopping-time result can be made rigorous, the paper would be a valuable contribution to sequential analysis and anomaly detection. However, the current manuscript does not yet supply a complete proof of the headline bound, because the Markov regret assumption is not formally defined and the key pathwise domination step is not justified.
major comments (4)
- [Definition 2, §III-B and §III-C] Definition 2 defines the cumulative pointwise worst-case regret only for i.i.d. distributions q ∈ Δ^{m−1}. Theorem 2, however, invokes an analogous regret guarantee for Markov estimators {\hat{Q}_t} without formally defining the Markov version or stating over which class of alternatives the supremum is taken. Without a deterministic, pathwise Markov regret bound—i.e., a pointwise inequality of the form log L*_t − log L_t ≤ r_t holding for every sample path and every t—the inequality τ ≤ τ* in §III-C does not follow from the stated assumptions. The paper needs to define the Markov regret precisely and either prove the O(log t) pointwise bound for the Jeffreys mixture estimator or cite a theorem in [9] that supplies it.
- [§III-C, Eq. (4)] The displayed derivation leading to Eq. (4) contains a non-sequitur that is load-bearing for the expected stopping time bound. The proof defines τ* via an oracle statistic L*_t, then writes "E[log L_{τ*}] ≤ log 1/α + E[log τ*] + C". This mixes L and L*: the oracle stopping time controls log L*_{τ*}, not log L_{τ*}. Furthermore, the overshoot term is written as E[log L*_τ − log L_{τ*−1}], which should be E[log L*_{τ*} − log L*_{τ*−1}]. The intended inequality is likely E[log L*_{τ*}] ≤ log(1/α) + E[r_{τ*}] + O(1), with r_t = O(log t) converted to E[log τ*] via Jensen; this needs to be written out correctly, and the uniform bound on the overshoot needs a proof rather than an assertion.
- [Remark 4, §III-C] Remark 4 states that the bounded-likelihood assumption max_{i,j} Q(j|i)/P(j|i) ≤ c' < ∞ can be removed with "a slightly more involved argument", but no proof or reference is given. Since the stated Theorem 2 uses c' to bound the per-step log-likelihood increments and to apply the Markov Wald identity, the advertised claim in the Introduction that no assumption is made on the alternative is not supported by the theorem as written. The authors should either prove the c' < ∞ case rigorously with a finite overshoot argument, or temper the claims in the abstract and introduction to reflect the condition actually used.
- [§III-D, Jeffreys mixture claim] The paper asserts that the Jeffreys mixture estimator for Markov chains attains regret O((m−1)m/2 log(t/2π)) and cites [9], but does not specify whether the cited result is pointwise (pathwise) or in expectation and does not give a theorem number. This matters because the proof of Theorem 2 requires a pointwise deterministic regret bound for every sample path, not merely an averaged or high-probability bound. The authors should state the precise regularity and uniformity of the regret bound obtained from [9] or prove the needed pathwise bound; otherwise the substitution of r_t in the oracle-stopping-time argument is unjustified.
minor comments (5)
- [§III-A] There is a typo in "Our alogorithm and test work with any method..." — "alogorithm" should be "algorithm".
- [§III-C, proof of Theorem 2] In the definition of τ*, the right-hand side is written as a set ("≤ {t ≥ 1 : ... } =: τ*"), but τ* should be defined as the infimum of that set, i.e., τ* := inf{t ≥ 1 : log L*_t ≥ log(1/α) + r_t}.
- [§III-C, martingale check] In the display verifying the martingale property, the summation index is misprinted as "∑_{i=j}"; it should be "∑_{j∈[m]}".
- [Eq. (8) and Fig. 2] The caption of Fig. 2 refers to "Problem 8", which is likely Eq. (8); please make the cross-reference explicit and consistent.
- [Definition 2] The notation "r_t((\hat{q}_i)^t_{i=1})" is awkward: the subscript t on the left is redundant, and the definition as a maximum over trajectories and a supremum over q is not clearly separated. A cleaner notation would help the reader distinguish the pointwise regret from its worst-case bound.
Circularity Check
No circular derivation: the main theorem is a conditional reduction from estimator regret to SPRT-matching stopping time, with independent external ingredients.
full rationale
The paper's central claim is Theorem 2: if a causal estimator has worst-case pointwise regret r_t = O(log t), then the proposed one-sided SPRT-type test has type-1 error at most alpha and expected stopping time O(log(1/(alpha D_M(Q||P)))/D_M(Q||P)). The type-1 bound follows from Ville's inequality applied to the nonnegative martingale L_t, and the expected-stopping-time bound follows by comparing L_t with the oracle likelihood ratio L*_t at a cost of the estimator's regret, then applying a Markov-chain Wald identity from Moustakides to evaluate E[log L*_{tau*}]. Neither step assumes the conclusion: the regret bound is an explicit input condition, and the SPRT-matching constant D_M(Q||P) enters through the independent Wald identity rather than through the algorithm's construction. The existence of estimators with the required regret is cited to Takeuchi, Kawabata, and Barron, with no author overlap with the present paper, so there is no self-citation chain. No parameter is fitted and later reported as a prediction, and no quantity is defined in terms of the stopping time it is supposed to explain. The manuscript does contain rigor concerns that are not circularity: Definition 2 states pointwise worst-case regret in the i.i.d. setting and the Markov analogue is only asserted via citation, and Remark 4 claims the c' < infinity likelihood-ratio assumption can be removed without proof. These affect proof completeness, not whether the derivation reduces to its own inputs. Therefore the circularity score is 0.
Assumptions & free parameters
assumptions (5)
- standard math Ville's inequality for nonnegative supermartingales.
- domain assumption Wald's identity for Markov chains (Moustakides 1999).
- domain assumption Regret bound O(log t) for the chosen estimator, instantiated by Takeuchi and Barron's Jeffreys mixture.
- domain assumption The alternative chain Q is ergodic and satisfies max_{i,j} Q(j|i)/P(j|i) <= c' for a finite constant c'.
- domain assumption The estimator Q-hat_t is a valid probability distribution adapted to the observed history.
Cite this review
Pith. "Pith review of Sequential One-Sided Hypothesis Testing of Markov Chains." pith.science (2026). https://pith.science/paper/GFIZRRJN
@misc{pith2026250113187,
author = {Pith},
title = {Pith review of: Sequential One-Sided Hypothesis Testing of Markov Chains},
year = {2026},
howpublished = {\url{https://pith.science/paper/GFIZRRJN}},
note = {Machine review of arXiv:2501.13187}
}
abstract
We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory of a Markov chain with a known transition matrix $P$ (null hypothesis) or not (composite alternative hypothesis). This problem naturally arises in many engineering problems. The main technical challenge is to develop a sequential testing scheme that adapts its sample size to the unknown alternative. Indeed, if we knew the alternative distribution (that is, the transition matrix) $Q$, a natural approach would be to use a generalization of Wald's sequential probability ratio test (SPRT). Building on this intuition, we propose and analyze a family of one-sided SPRT-type tests for our problem that use a data-driven estimator $\hat{Q}$. In particular, we show that if the deployed estimator admits a worst-case regret guarantee scaling as $\mathcal{O}\left( \log{t} \right)$, then the performance of our test asymptotically matches that of SPRT in the simple hypothesis testing case. In other words, our test automatically adapts to the unknown hardness of the problem, without any prior information. We end with a discussion of known Markov chain estimators with $\mathcal{O}\left( \log{t} \right)$ regret.
Figures
Forward citations
Cited by 1 Pith paper
-
Asymptotically Optimal Sequential Testing with Markovian Data
For ergodic finite-state Markov chains, the optimal expected stopping time of an α-correct one-sided sequential test is asymptotically log(1/α)/Dinf_M(Q,P), where Dinf is the least stationary-weighted KL divergence fr...
Reference graph
Works this paper leans on
-
[9]
Properties o f Jeffreys mixture for Markov sources,
J. Takeuchi, T. Kawabata, and A. R. Barron, “Properties o f Jeffreys mixture for Markov sources,” IEEE transactions on information theory , vol. 59, no. 1, pp. 438–457, 2012
work page 2012
-
[1]
Large deviations, hypotheses testing, a nd source coding for finite Markov chains,
S. Natarajan, “Large deviations, hypotheses testing, a nd source coding for finite Markov chains,” IEEE Trans. Inf. Theory, vol. 31, pp. 360–365, 1985. [Online]. Available: https://api.semanticscholar.org/CorpusID:27423979
work page 1985
-
[2]
Sequential Analysis of Dependent Obse rvations. I,
R. M. Phatarfod, “Sequential Analysis of Dependent Obse rvations. I,” Biometrika, vol. 52, no. 1/2, pp. 157–165, 1965. [Online]. Available: http://www.jstor.org/stable/2333821
-
[3]
Sequential Tests of Statistical Hypotheses,
A. Wald, “Sequential Tests of Statistical Hypotheses,” The Annals of Mathematical Statistics , vol. 16, no. 2, pp. 117 – 186, 1945. [Online]. Available: https://doi.org/10.1214/aoms/1177731118
arXiv 1945
-
[4]
Minimax optimal se quential hypothesis tests for Markov processes,
M. Fauss, A. M. Zoubir, and H. V . Poor, “Minimax optimal se quential hypothesis tests for Markov processes,” The Annals of Statistics , vol. 48, no. 5, pp. 2599–2621, 2020
work page 2020
-
[5]
Asymptotically optimal tests for multin omial distribu- tions,
W. Hoeffding, “Asymptotically optimal tests for multin omial distribu- tions,” The Annals of Mathematical Statistics , pp. 369–401, 1965
work page 1965
-
[6]
Generalized sequential proba bility ratio test for separate families of hypotheses,
X. Li, J. Liu, and Z. Ying, “Generalized sequential proba bility ratio test for separate families of hypotheses,” Sequential analysis, vol. 33, no. 4, pp. 539–563, 2014
work page 2014
-
[7]
Statistical anomaly det ection via composite hypothesis testing for markov models,
J. Zhang and I. C. Paschalidis, “Statistical anomaly det ection via composite hypothesis testing for markov models,” IEEE Transactions on Signal Processing , vol. 66, no. 3, pp. 589–602, 2018
work page 2018
Show all 22 references
-
[8]
Sequential algorithms for testing identity and closeness of distributions,
O. Fawzi, N. Flammarion, A. Garivier, and A. Oufkir, “Sequential algorithms for testing identity and closeness of distributions,” ArXiv, vol. abs/2205.06069, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:248721783
2022 arXiv
-
[10]
Minimax Testing of Ident ity to a Reference Ergodic Markov Chain,
G. Wolfer and A. Kontorovich, “Minimax Testing of Ident ity to a Reference Ergodic Markov Chain,” in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , ser. Proceedings of Machine Learning Research, S. Chiappa a nd R. Calandra, ...
2020
-
[11]
A survey on distribution testing: Y our d ata is big. but is it blue?
C. L. Canonne, “A survey on distribution testing: Y our d ata is big. but is it blue?” Theory of Computing , pp. 1–100, 2020
2020
-
[12]
Exact goodness-of-fit tests for markov chains,
J. Besag and D. Mondal, “Exact goodness-of-fit tests for markov chains,” Biometrics, vol. 69, no. 2, pp. 488–496, 2013
2013
-
[13]
Testing for the markov property in t ime series,
B. Chen and Y . Hong, “Testing for the markov property in t ime series,” Econometric Theory , vol. 28, no. 1, pp. 130–178, 2012
2012
-
[14]
Extension of Wald’s first lemma to Ma rkov pro- cesses,
G. V . Moustakides, “Extension of Wald’s first lemma to Ma rkov pro- cesses,” Journal of applied probability , vol. 36, no. 1, pp. 48–59, 1999
1999
-
[15]
On Learning Distributions from their Samples,
S. Kamath, A. Orlitsky, D. Pichapati, and A. T. Suresh, “ On Learning Distributions from their Samples,” in Proceedings of The 28th Conference on Learning Theory , ser. Proceedings of Machine Learning Research, P . Gr¨ unwald, E. Hazan, and S. Kale, Eds., vol. 40. Paris, Franc...
2015
-
[16]
The performance of unive rsal encod- ing,
R. Krichevsky and V . Trofimov, “The performance of unive rsal encod- ing,” IEEE Transactions on Information Theory , vol. 27, no. 2, pp. 199– 207, 1981
1981
-
[17]
An automatic inequality pro ver and instance optimal identity testing,
G. V aliant and P . V aliant, “An automatic inequality pro ver and instance optimal identity testing,” SIAM Journal on Computing , vol. 46, no. 1, pp. 429–455, 2017
2017
-
[18]
Bayesian context trees: Modelling and exa ct inference for discrete time series,
I. Kontoyiannis, L. Mertzanis, A. Panotopoulou, I. Pap ageorgiou, and M. Skoularidou, “Bayesian context trees: Modelling and exa ct inference for discrete time series,” Journal of the Royal Statistical Society Series B: Statistical Methodology , vol. 84, no. 4, pp. 1287–1323, 2022
2022
-
[19]
DeepAR: Probabilistic forecasting with autoregressive recurrent networks,
D. Salinas, V . Flunkert, J. Gasthaus, and T. Januschows ki, “DeepAR: Probabilistic forecasting with autoregressive recurrent networks,” Inter- national journal of forecasting , vol. 36, no. 3, pp. 1181–1191, 2020
2020
-
[20]
A survey of methods fo r time series change point detection,
S. Aminikhanghahi and D. J. Cook, “A survey of methods fo r time series change point detection,” Knowledge and information systems , vol. 51, no. 2, pp. 339–367, 2017
2017
-
[21]
Chang e-point detection and segmentation of discrete data using bayesian context tr ees,
V . Lungu, I. Papageorgiou, and I. Kontoyiannis, “Chang e-point detection and segmentation of discrete data using bayesian context tr ees,” arXiv preprint arXiv:2203.04341, 2022
2022 arXiv
-
[22]
Asymptotic operating characteristics of a n optimal change point detection in hidden markov models,
C.-D. Fuh, “Asymptotic operating characteristics of a n optimal change point detection in hidden markov models,” Annals of Statistics , pp. 2305–2339, 2004
2004
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.