pith. sign in

arxiv: 1907.01582 · v1 · pith:2Q4A4VNUnew · submitted 2019-07-02 · ❄️ cond-mat.stat-mech · cs.IT· math.IT· physics.bio-ph

Minimum Power to Maintain a Nonequilibrium Distribution of a Markov Chain

Dmitri S. Pavlichin , Yihui Quek , Tsachy Weissman This is my paper

Pith reviewed 2026-05-25 10:26 UTC · model grok-4.3

classification ❄️ cond-mat.stat-mech cs.ITmath.ITphysics.bio-ph
keywords Markov chainstationary distributionKL divergence ratenonequilibriumpower costcontrolbirth-death processreversible chain
0
0 comments X

The pith

The minimal power to hold a Markov chain in a target stationary distribution is the minimal KL divergence rate from the uncontrolled chain.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Biological systems spend energy to sustain non-equilibrium distributions of states over long times. The paper models this as altering an uncontrolled Markov chain Q into a controlled chain P that has a prescribed stationary distribution. The thermodynamic cost of this control is taken to be the Kullback-Leibler divergence rate D(P||Q). The central result is that the chain P* minimizing this rate under the stationary-distribution constraint gives the lowest possible cost, which therefore lower-bounds the power required. Explicit solutions are derived for reversible Q, two-state chains, and birth-and-death processes, in both discrete and continuous time.

Core claim

The optimal controlled chain P* minimizes the KL divergence rate D(·||Q) subject to a stationary distribution constraint, and the minimal KL divergence rate lower bounds the power used. For a reversible uncontrolled chain Q the minimizer admits an explicit form; similar closed expressions hold for two-state chains and birth-and-death processes.

What carries the argument

The KL divergence rate D(P||Q) between controlled and uncontrolled transition kernels, minimized subject to the constraint that P has a prescribed stationary distribution.

If this is right

  • The minimal power cost equals the minimized KL rate for any chosen stationary distribution.
  • Closed-form expressions exist when the uncontrolled chain Q is reversible.
  • The same minimization yields explicit solutions for two-state chains and birth-and-death processes.
  • The result holds for both discrete-time and continuous-time Markov chains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bound supplies a quantitative efficiency limit that could be compared against measured energy use in synthetic molecular circuits.
  • Design of low-power controllers for stochastic systems may be guided by constructing the optimal P* rather than heuristic policies.
  • The same variational problem appears in large-deviations theory; the thermodynamic reading may suggest new rate-function interpretations in other controlled stochastic processes.

Load-bearing premise

The thermodynamic cost of steering the chain is exactly captured by the KL divergence rate between the controlled and uncontrolled transition probabilities.

What would settle it

Construct or simulate a physical Markov process whose control cost can be measured directly and check whether the measured power ever falls below the computed minimal KL rate for the same target stationary distribution.

Figures

Figures reproduced from arXiv: 1907.01582 by Dmitri S. Pavlichin, Tsachy Weissman, Yihui Quek.

Figure 1
Figure 1. Figure 1: Uncontrolled chain Q (a toy model for the slack arm pulled down by gravity, with smaller-index states closer to the ground and rates q− > q+): the birth-and-death chain with 5 states in continuous time. The states correspond to minima in the potential landscape experienced by a single protein at different positions along the muscle fiber. distribution π ∗ . A computation shows the 2 × 2 minimum￾power trans… view at source ↗
Figure 3
Figure 3. Figure 3: The minimum-power discrete-time controlled chain [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (top) Time evolution of the |X | = 5 components of µt = µ0(P ∗ ) t with µ0 = π, where P ∗ is the minimum cost chain (26), showing µt → π ∗ as t → ∞. (bottom) The cost (blue) Dµt (P ∗ ||Q) (20) and the minimum power (red) D(P ∗ ||Q). independent force upwards (away from state 1) as in the continuous time case. V. DISCUSSION This work derives the minimum power required to maintain a target stationary distrib… view at source ↗
Figure 5
Figure 5. Figure 5: (top) A gas molecule is found underneath the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
read the original abstract

Biological systems use energy to maintain non-equilibrium distributions for long times, e.g. of chemical concentrations or protein conformations. What are the fundamental limits of the power used to "hold" a stochastic system in a desired distribution over states? We study the setting of an uncontrolled Markov chain $Q$ altered into a controlled chain $P$ having a desired stationary distribution. Thermodynamics considerations lead to an appropriately defined Kullback-Leibler (KL) divergence rate $D(P||Q)$ as the cost of control, a setting introduced by Todorov, corresponding to a Markov decision process with mean log loss action cost. The optimal controlled chain $P^*$ minimizes the KL divergence rate $D(\cdot||Q)$ subject to a stationary distribution constraint, and the minimal KL divergence rate lower bounds the power used. While this optimization problem is familiar from the large deviations literature, we offer a novel interpretation as a minimum "holding cost" and compute the minimizer $P^*$ more explicitly than previously available. We state a version of our results for both discrete- and continuous-time Markov chains, and find nice expressions for the important case of a reversible uncontrolled chain $Q$, for a two-state chain, and for birth-and-death processes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper claims that the minimum power to maintain a desired nonequilibrium stationary distribution π in a controlled Markov chain P (derived from an uncontrolled chain Q) is lower-bounded by the minimal KL divergence rate D(P||Q) subject to the stationary-distribution constraint on P. It provides explicit constructions of the optimal P* for the cases of reversible Q, two-state chains, and birth-death processes, in both discrete and continuous time, interpreting the optimization (standard in large-deviations theory) as a minimum holding cost via the Todorov MDP framework.

Significance. If the thermodynamic identification of D(P||Q) as physical power cost holds, the results supply a concrete, computable lower bound on dissipation for sustaining nonequilibrium distributions together with closed-form minimizers for several important chain classes. The explicit solutions for reversible, two-state, and birth-death cases are a clear strength and distinguish the contribution from prior large-deviations literature.

minor comments (2)
  1. [Introduction] The abstract states that the minimizer is computed 'more explicitly than previously available'; a short paragraph in the introduction or §3 comparing the new closed forms to the expressions in the cited large-deviations references would make this improvement concrete.
  2. Notation for the stationary distribution π and the controlled transition kernel P is introduced gradually; defining both at the first appearance of the optimization problem (likely §2) would aid readability.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive report, recognition of the explicit solutions for reversible chains, two-state systems, and birth-death processes, and recommendation to accept the manuscript.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper attributes the KL rate cost function to Todorov's prior MDP framework and draws the optimization problem from large-deviations literature; neither is a self-citation. The central claim (minimum KL rate lower-bounds power under the modeling choice) follows directly from the stated interpretation without internal reduction to fitted parameters, self-definitional loops, or ansatzes. Explicit minimizers for reversible chains, two-state systems, and birth-death processes are derived from the optimization equations themselves. The derivation chain is self-contained against external benchmarks with no load-bearing self-citations or renamings of known results as novel derivations.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are identifiable beyond the standard definition of KL rate as control cost taken from prior literature.

pith-pipeline@v0.9.0 · 5763 in / 1093 out tokens · 28112 ms · 2026-05-25T10:26:55.466406+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages

  1. [1]

    Conditional limit theorems under Markov conditioning,

    I. Csiszár, T. M. Cover, and B.-S. Choi, “Conditional limit theorems under Markov conditioning,” IEEE Trans. Inf. Theory, vol. 33, pp. 788– 801, 1987

    work page 1987

  2. [2]

    R. P. Feynman, The Feynman lectures on physics . Addison-Wesley, 1964, vol. I

    work page 1964

  3. [3]

    Muscle structure and theories of contraction,

    A. F. Huxley, “Muscle structure and theories of contraction,” Prog. Biophys. Biophys. Chem. , vol. 7, pp. 257–318, 1957

    work page 1957

  4. [4]

    The mathematical theory of molecular motor movement and chemomechanical energy transduction,

    H. Qian, “The mathematical theory of molecular motor movement and chemomechanical energy transduction,” J. Math. Chem. , vol. 27, pp. 219–234, 2000

    work page 2000

  5. [5]

    Linearly-solvable Markov decision problems,

    E. Todorov, “Linearly-solvable Markov decision problems,” in Advances in Neural Information Processing Systems 19 , 2007, pp. 1369–1376

    work page 2007

  6. [6]

    Efficient computation of optimal actions,

    ——, “Efficient computation of optimal actions,” Proceedings of the National Academy of Sciences , vol. 106, pp. 11 478–11 483, 2009

    work page 2009

  7. [7]

    Minimum energetic cost to maintain a target nonequilibrium state,

    J. M. Horowitz, K. Zhou, and J. L. England, “Minimum energetic cost to maintain a target nonequilibrium state,” Phys. Rev. E, vol. 95, p. 042102, 2017

    work page 2017

  8. [8]

    Information-theoretic bound on the entropy production to maintain a classical nonequilibrium distribution using ancillary control,

    J. M. Horowitz and J. L. England, “Information-theoretic bound on the entropy production to maintain a classical nonequilibrium distribution using ancillary control,” Entropy, vol. 19, p. 333, 2017

    work page 2017

  9. [9]

    Dembo and O

    A. Dembo and O. Zeitouni, Large deviations techniques and applica- tions, 2nd edition . Springer-Verlag Berlin Heidelberg, 2010

    work page 2010

  10. [10]

    A representation formula for the large deviation rate function for the empirical law of a continuous time Markov chain,

    P. Baldi and M. Piccioni, “A representation formula for the large deviation rate function for the empirical law of a continuous time Markov chain,” Stat. & Prob. Letters , vol. 41, pp. 107–115, 1999

    work page 1999

  11. [11]

    Mézard and A

    M. Mézard and A. Montanari, Information, Physics, and Computation . Oxford University Press, 2009

    work page 2009

  12. [12]

    Kittel and H

    C. Kittel and H. Kroemer, Thermal Physics , 2nd ed. W. H. Freeman and Company, 1980

    work page 1980

  13. [13]

    E. L. L.D. Landau, Statistical Physics. Elsevier, 1951, vol. 5

    work page 1951

  14. [14]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, 623–656, 1948

    work page 1948

  15. [15]

    T. M. Cover and J. A. Thomas, Elements of Information Theory, Second Edition. Hoboken, NJ: John Wiley & Sons, 2006

    work page 2006

  16. [16]

    Relative entropy between Markov transition rate matrices,

    G. Kesidis and J. Walrand, “Relative entropy between Markov transition rate matrices,” IEEE Trans. Inf. Theory , vol. 39, pp. 1056–1057, 1993

    work page 1993

  17. [17]

    Large deviation principle for Markov chains in continuous time,

    A. de La Fortelle, “Large deviation principle for Markov chains in continuous time,” Probl. Inf. Trans., vol. 37, pp. 120–139, 2001

    work page 2001

  18. [18]

    From level 2.5 to level 2 large deviations for continuous time Markov chains,

    L. Bertini, A. Faggionato, and D. Gabrielli, “From level 2.5 to level 2 large deviations for continuous time Markov chains,” Markov processes and related fields, vol. 20, pp. 545–562, 2014

    work page 2014

  19. [19]

    A cost/speed/reliability tradeoff to erasing,

    M. Gopalkrishnan, “A cost/speed/reliability tradeoff to erasing,” Entropy, vol. 18, p. 165, 2016

    work page 2016

  20. [20]

    Modeling molecular motors,

    F. Jülicher, A. Ajdari, and J. Prost, “Modeling molecular motors,” Reviews of Modern Physics , vol. 69, pp. 1269–1281, 1997

    work page 1997

  21. [21]

    D. A. Levin, Y . Peres, and E. L. Wilmer, Markov chains and mixing times. American Mathematical Society, 2009

    work page 2009

  22. [22]

    On the decrease of entropy in a thermodynamic system by the intervation of intelligent beings,

    L. Szilard, “On the decrease of entropy in a thermodynamic system by the intervation of intelligent beings,” Zeitschrift für Physik , vol. 53, pp. 840–856, 1929. 𝑉𝑖 ′ pistons move 𝑉𝑖 𝑖 Fig. 5: (top) A gas molecule is found underneath the i-th piston with probability proportional to volume Vi. (bottom) Inserting impermeable partitions between the pistons, ...

    work page 1929