Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

On Limiting Probability Distributions of Higher Order Markov Chains

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A higher-order Markov chain whose transition tensor is regular has a unique limiting probability distribution: the tensor powers converge to a positive rank-one limit, and the state probabilities converge to that limit regardless of the…

desk verdict Solid P>0 case and useful Q-eigenvector connection, but the regular-case proof of Theorem 3.3 has a load-bearing gap. read the letter →

arxiv 2506.08874 v2 pith:RUZC3DJT submitted 2025-06-10 math.PR math.STstat.TH

Lixing Han , Jianhong Xu This is my paper
classification math.PRmath.STstat.TH MSC 60J1015A69
keywords higherorderMarkovchainslimitingprobabilitydistributionregulartransitiontensorpowersreducedfirstchaindominanteigenvectorsmatricizationstationary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to prove that a higher-order Markov chain has a true limiting probability distribution whenever its transition tensor is regular, meaning some power of the tensor has all entries positive. If correct, this extends the classical first-order Markov chain theorem about regular transition matrices to chains with memory of length $m-1$, without relying on approximation schemes or two-phase power iterations. The authors also show that the limiting probabilities can be recovered from an eigenvector of the reduced first-order chain, even in cases where that reduced chain is itself not regular. The upshot is a sharper and more general picture of when long-run behavior of a higher-order chain is well defined.

What carries the argument

The main workhorse is the mode-1 matricization of the transition tensor, which flattens the tensor into a matrix by stacking frontal slices side by side, together with the tensor-power recurrence $P^{(k+1)}=P^{(k)}\boxtimes P$. Lemma 3.1, $P^{(u+v)}=P^{(u)}Q^v$, links tensor powers of $P$ to ordinary matrix powers of the reduced first-order chain $Q$, allowing regularity of $P$ to be read off from $Q$ and allowing the limiting tensor to be expressed through eigenvectors of $Q$. The convergence proof itself is a contraction argument: tracking the maximum $U_k$ and minimum $L_k$ of entries of $P^{(k)}$ along mode-1 fibers, the positive-tensor case shows $U_k-L_k$ shrinks by a factor $(1-2\epsilon^{m-1})$ at lags $m-1,2m-1,3m-2,\dots$; Theorem 3.3 asserts the same argument applies to a regular tensor by replacing $\epsilon$ with $\min_{i_1\dots i_m} p^{(K)}_{i_1\dots i_m}$. Theorem 3.4 completes the link from the rank-one tensor limit to marginal probabilities by summing over all joint initial histories.

What would settle it

Compute the tensor powers of a small regular transition tensor, for example a third-order chain with $n=3$ and $K=2$, and check numerically whether the fiberwise spread $\max_{i_2\dots i_m} p^{(k)}_{i i_2\dots i_m}-\min_{i_2\dots i_m} p^{(k)}_{i i_2\dots i_m}$ converges to zero; the theorem would be false if a regular $P$ produced a periodic or nonconvergent sequence of powers, or if two different normalized nonnegative eigenvectors $y$ of $Q$ with $Qy=y$ gave different values of $P^{(0)}y$ in a case where $P$ is regular.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is Theorem 3.3: if the $m$th-order transition tensor $P$ is regular, so some tensor power $P^{(K)}$ is entrywise positive, then the tensor powers converge to a rank-one limit $\lim_{k\to\infty} P^{(k)}=\pi\otimes e\otimes\cdots\otimes e$, where $\pi$ is a strictly positive probability vector. Theorem 3.4 turns this into a statement about the chain's marginals: for every state $i$, $\lim_{t\to\infty}\Pr(X_t=i)=\pi_i$, and this limit does not depend on the initial probability distributions $x_1,\dots,x_{m-1}$. The paper additionally proves that regularity of the reduced first-order transition matrix $Q$ implies regularity of $P$ (Theorem 3.1) and that for $P>0$ the reduced chain $Q$ is regular (Theorem 3.2). Finally, Theorem 3.6 identifies the limiting vector as $\pi=P^{(0)}y$ for any normalized nonnegative right eigenvector $y$ of $Q$ with eigenvalue $1$, and Example 3.4 shows this identification remains valid when $Q$ is non-regular and has a two-dimensional dominant eigenspace.

Load-bearing premise

The proof that the contraction argument extends from strictly positive tensors to merely regular tensors is asserted without details; the load-bearing premise is that taking subsequences at multiples of K preserves the same coefficient lower bounds, even though the reduced chain Q can be non-regular in that setting.

Editorial extensions

If this is right

  • If $P$ is regular, its tensor powers converge to $\pi\otimes e\otimes\cdots\otimes e$ with $\pi>0$, so long-run state probabilities are well defined.
  • The limiting marginal probabilities are independent of how the chain is started, matching the first-order regular-chain guarantee.
  • Regularity of the reduced first-order chain $Q$ is a sufficient way to certify regularity of $P$, so standard matrix regularity tests can be reused.
  • When a limiting distribution exists, it can be obtained from any normalized nonnegative dominant eigenvector of $Q$ via $\pi=P^{(0)}y$, even if $Q$ is non-regular and the dominant eigenspace has dimension greater than one.
  • Ergodicity alone does not guarantee a limiting distribution; the paper gives an ergodic non-regular chain whose marginals oscillate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The unproved step in the regular case suggests a concrete numerical check: compute the lags $U_{mK}-L_{mK}$ for a regular tensor to see whether the contraction factor from the positive case actually holds; this would either close the gap or reveal the need for a different argument.
  • Because Theorem 3.4 does not require $\pi>0$, the same framework may describe absorbing higher-order chains where some states have limiting probability zero; one could test this on a small absorbing example.
  • The eigenvector recipe $\pi=P^{(0)}y$ suggests a projection-type estimator for limiting probabilities that works when power iteration on $Q$ diverges, which could be tested against the example in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies the limiting probability distribution of a homogeneous (m-1)th order Markov chain on a finite state space. The central result, Theorem 3.3, asserts that if the transition tensor P is regular in the sense that some power P^(K) has all entries positive, then the k-step transition tensors converge to pi ⊗ e ⊗ ... ⊗ e with pi > 0, independent of the initial distributions; Theorem 3.4 transfers this to convergence of the marginal distributions x_t. The paper also proves a lemma relating matricized tensor powers to powers of the reduced first-order chain Q, a theorem showing P > 0 implies Q regular, a tightness example in which ergodicity without regularity fails to yield a limit, and an eigenvector identity pi = P^(0)y that remains valid when Q is non-regular. Several illustrative examples are provided.

Significance. If Theorem 3.3 is correct, the paper makes a useful contribution: it gives an exact, approximation-free sufficient condition for existence of a limiting probability distribution of a higher-order Markov chain, generalizing the first-order regular matrix case and going beyond approaches that require the reduced chain Q to be regular. The contraction proof for the P > 0 case is detailed and internally consistent, and Example 3.4 is a valuable demonstration that Q can be non-regular even when P is regular. The main claims are stated cleanly and the examples are informative. However, the extension from P > 0 to regular P is not justified in the text, and because Theorem 3.3 is the source of the main conclusions, the manuscript is not yet fully established.

major comments (2)
  1. [§3, Theorem 3.3, final paragraph (Eq. (3.6))] The proof of the regular case is not a routine extension of the P > 0 case. In the positive case the contraction factor 1 - 2ε^(m-1) is obtained by iterating (3.6) m-1 times, where every coefficient p_{j i2...im} is at least ε. If P is only regular, replacing ε by min p^(K) would require a block analogue of (3.6) whose coefficients are the K-step probabilities p^(K)_{...}; the manuscript proves no such recurrence. The coefficients that actually appear by iterating (3.6) K times are entries of Q^K, not entries of P^(K), and by the authors' own Example 3.4, Q^K need not be positive when P is regular. Hence the asserted subsequence U0-L0, UK-LK, UmK-LmK, ... has no demonstrated contraction, and convergence of U_k-L_k to zero is not established. Because Theorems 3.4 and 3.6 rely on Theorem 3.3, this gap is load-bearing. The authors should either supply a complete argument for the regular case or restrict the theorem to the positive case until such an argument is available.
  2. [§3, Theorem 3.3, final paragraph] The phrase "The proof can be done in exactly the same fashion" is also problematic for a structural reason. One might try to apply the positive-case proof to the positive tensor P^(K) itself, but that requires an identity of the form (P^(K))^(r) = P^(rK). No such identity is stated or proved, and since Section 2 notes that the tensor product ⊠ is neither associative nor commutative, the identity is not automatic; the recurrence (2.1) only provides P^(k+1) = P^(k) ⊠ P. Thus a reader cannot infer the inequality U_{k+K} - L_{k+K} ≤ (1 - 2ε^(m-1))(U_k - L_k) from the displayed argument. Please add the missing recurrence or a different substitution.
minor comments (4)
  1. [Abstract] "Several illustrative example are also given" should read "Several illustrative examples are also given."
  2. [§2, after Eq. (2.1)] Because ⊠ is non-associative, the phrase "P^(k) can be thought of as the kth power of P" should be clarified: it is the recursively defined k-step transition tensor, not a k-fold product in the usual sense. An explicit statement would prevent confusion in the proof of Theorem 3.3.
  3. [§3, Example 3.4] The example asserts P^(10) > 0 and describes the double dominant eigenvalue and eigenvectors of Q without showing the computation. Since the example is used to show that Theorem 3.3 applies beyond the Q-regular case, please provide the verification or a precise reference.
  4. [§3, after Example 3.2] The word "tight" is stronger than the example supports: Example 3.2 shows that ergodicity without regularity can fail to yield a limiting distribution, but it does not show that regularity is necessary for the existence of such a distribution. Please rephrase the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning: Theorem 3.3 and its consequences are derived from contraction arguments over the transition tensor; prior self-citations are background only.

full rationale

The paper's central claim, Theorem 3.3, is proved by a direct epsilon-contraction argument starting from the definition of P(k) and recurrence (2.1), without assuming the target result. The P > 0 case is self-contained, and the regular case is asserted by analogy through the subsequence U0 - L0, UK - LK, etc.; that step has a nontrivial proof gap because positivity of P(K) does not by itself give the coefficient lower bound needed for contraction, but a proof gap is not circularity since the asserted extension is not obtained by assuming the theorem or by renaming an input. Theorem 3.4 follows from Theorem 3.3 by total probability, and Theorem 3.6 uses Lemma 3.1, which is proved from definitions. Citations to the authors' own prior work ([9], [10], [11]) occur only in background statements about ergodicity, passage times, and state classification, and are not load-bearing for the limiting-distribution theorems. No parameter is fitted and no definition is made in terms of the quantity being derived. Therefore the analysis finds no circular step; the central derivation is independent, with the noted regularity-case proof being incomplete rather than circular.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The proof rests on definitions and a proven lemma. No constants are fitted to data, and no new states or parameters are introduced beyond the known reduced first order chain. The main tacit assumption is that the contraction argument extends to regular tensors; this is flagged in red flags and in weakest_assumption.

assumptions (3)
  • domain assumption The process is a homogeneous finite-state higher order Markov chain satisfying the defining conditional independence (1.1).
    All results are stated for this model; the paper does not address non-homogeneous or infinite-state chains.
  • standard math The transition tensor is stochastic: entries are nonnegative and each mode-1 column sums to 1.
    Used to define k-step transition probabilities and to guarantee that convex combinations in the contraction proof are probability mixtures.
  • standard math The recurrence (2.1) correctly defines k-step transition probabilities and identity (3.1) connects tensor powers with powers of the reduced chain matrix Q.
    Lemma 3.1 is proven in the paper, but the proof relies on the Markov property and on the construction of Q; these are standard.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Limiting Probability Distributions of Higher Order Markov Chains." pith.science (2026). https://pith.science/paper/RUZC3DJT

@misc{pith2026250608874,
  author       = {Pith},
  title        = {Pith review of: On Limiting Probability Distributions of Higher Order Markov Chains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RUZC3DJT}},
  note         = {Machine review of arXiv:2506.08874}
}
read the original abstract

The limiting probability distribution is one of the key characteristics of a Markov chain since it shows its long-term behavior. In this paper, for a higher order Markov chain, we establish some properties related to its exact limiting probability distribution, including a sufficient condition for the existence of such a distribution. Our results extend the corresponding conclusions on first order chains. Besides, they complement the existing results concerning higher order chains which rely on approximation schemes or two-phase power iterations. Several illustrative example are also given.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HOMC: A MATLAB Package for Higher Order Markov Chains

    stat.CO 2025-10 conditional novelty 5.0 of 10

    HOMC is a MATLAB package for higher-order Markov chains that computes transition power tensors, limiting distributions, ever-reaching probabilities, and mean first passage times.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bozorgmanesh and M

    H. Bozorgmanesh and M. Hajarian, Convergence of a transition proba- bility tensor of a higher-order Markov chain to the stationary probability vector, Numerical Linear Algebra with Applications 23 (2016): 972–988

    work page 2016

  2. [2]

    Chang, T

    K.C. Chang, T. Zhang, On the uniqueness and non-uniqueness of the positive Z-eigenvector for transition probability tensors, Journal of Math- ematical Analysis and Applications 408 (2013): 525–540

    work page 2013

  3. [3]

    J. Culp, K. Pearson, T. Zhang, On the uniqueness of the Z1-eigenvector of transition probability tensors, Linear and Multilinear Algebra 65 (2017): 891–896

    work page 2017

  4. [4]

    W. Ding, M. Ng, and Y. Wei, Fast computation of stationary joint proba- bility distribution of sparse Markov chains, Applied Numerical Mathemat- ics 125 (2018): 68–85

    work page 2018

  5. [5]

    Fasino, F

    D. Fasino, F. Tudisco, Ergodicity coefficients for higher-order stochastic processes, SIAM Journal on Mathematics of Data Science 2 (2020): 740– 769

    work page 2020

  6. [6]

    Gautier, F

    A. Gautier, F. Tudisco, M. Hein, A unifying Perron-Frobenius theorem for nonnegative tensors via multihomogeneous maps, SIAM Journal on Matrix Analysis and Applications 40 (2019): 1206–1231

    work page 2019

  7. [7]

    Geiger, A sufficient condition for a unique invariant distribution of a higher-order Markov chain, Statistics and Probability Letters 130 (2017): 49–56

    B.C. Geiger, A sufficient condition for a unique invariant distribution of a higher-order Markov chain, Statistics and Probability Letters 130 (2017): 49–56. 20

    work page 2017

  8. [8]

    Gleich, L.H

    D.F. Gleich, L.H. Lim, and Y. Yu, Multilinear pagerank, SIAM Journal on Matrix Analysis and Applications 36 (2015): 1507–1541

    work page 2015

Show all 26 references
  1. [9]

    L. Han, K. Wang, and J. Xu, Higher order ergodic Markov chains and first passage times, Linear and Multilinear Algebra 70 (2022): 6772–6779

    2022

  2. [10]

    Han and J

    L. Han and J. Xu, Ever-reaching probabilities and mean first passage times of higher order ergodic Markov chains, Linear and Multilinear Alge- bra 72 (2024): 59–75

    2024

  3. [11]

    Han and J

    L. Han and J. Xu, On classification of states in higher order Markov chains, Linear Algebra and Its Applications 685 (2024): 24–45

    2024

  4. [12]

    R. Horn, C. Johnson, Matrix Analysis , 1st ed., Cambridge University Press, Cambridge, 1985

    1985

  5. [13]

    Hu and L

    S. Hu and L. Qi, Convergence of a second order Markov chain, Applied Mathematics and Computation , 241 (2014), 183–192

    2014

  6. [14]

    Huang and L

    Z.H. Huang and L. Qi, Stationary probability vectors of higher-order two-dimensional symmetric transition probability tensors, Asia-Pacific Journal of Operational Research 37 (2020): 2040019, 14 pages

    2020

  7. [15]

    Hunter, Mathematical Techniques of Applied Probability, Vol

    J.J. Hunter, Mathematical Techniques of Applied Probability, Vol. 1, Discrete Time Models: Basic Theory , Academic Press, New York, 1983

    1983

  8. [16]

    Iosifescu, Finite Markov Processes and Their Applications , Dover Publications, New York, 2007

    M. Iosifescu, Finite Markov Processes and Their Applications , Dover Publications, New York, 2007

    2007

  9. [17]

    Kemeny, J.L

    J.G. Kemeny, J.L. Snell, Finite Markov Chains , Springer-Verlag, New York, 1960

    1960

  10. [18]

    Kilmer, C.D

    M.E. Kilmer, C.D. Martin, Factorization strategies for third-order ten- sors, Linear Algebra and Its Applications 435 (2011): 641–658

    2011

  11. [19]

    Kolda, B.W

    T.G. Kolda, B.W. Bader, Tensor decompositions and applications, SIAM Review 51 (2009): 455–500

    2009

  12. [20]

    Li and S

    C.K. Li and S. Zhang, Stationary probability vectors of higher-order Markov chains, Linear Algebra and Its Applications 473 (2016): 114–125. 21

    2016

  13. [21]

    Li and M

    W. Li and M. Ng, On the limiting probability distribution of a transition probability tensor, Linear and Multilinear Algebra 62 (2014): 362–385

    2014

  14. [22]

    Martin, R

    C.D. Martin, R. Shafer, B. Larue, An order- p tensor factorization with applications in imaging, SIAM Journal on Scientific Computing 35 (2013): 474–490

    2013

  15. [23]

    Qi and Z

    L. Qi and Z. Luo, Tensor Analysis: Spectral Theory and Special Tensors, SIAM, Philadelphia, 2017

    2017

  16. [24]

    Smilde, R

    A. Smilde, R. Bro, P. Geladi, Multi-Way Analysis: Applications in the Chemical Sciences, Wiley, West Sussex, UK, 2004

    2004

  17. [25]

    Vladimirescu, Regular homogeneous Markov chains of order two, Analele Universitatii din Craiova, Seria Matematica, Fizica-Chimie 13 (1985): 59–63 (Romanian)

    I. Vladimirescu, Regular homogeneous Markov chains of order two, Analele Universitatii din Craiova, Seria Matematica, Fizica-Chimie 13 (1985): 59–63 (Romanian)

    1985

  18. [26]

    Wu and M.T

    S.J. Wu and M.T. Chu, Markov chains with memory, tensor formulation, and the dynamics of power iteration, Applied Mathematics and Computa- tion 303 (2017): 226–239. 22

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.