The derivation of Markov processes that violate detailed balance

Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with detailed balance, are often used to model open systems driven out of equilibrium by external agents. I show that for a Markov model without detailed balance, an extended Markov model that explicitly includes the degrees of freedom for the driving agent can be constructed, such that the original cyclic Markov model for the driven system can be recovered as an approximation at early times, by summing over the degrees of freedom for the driving agent. In the process, the widely accepted formula for the entropy production in a cyclic Markov model is explicitly expressed as a time derivative of an entropy component in the extended model. I also find an analytic formula for the entropy component that is hidden in the cyclic Markov model.