An iterative approximation scheme for repetitive Markov processes

Tüfekçi, Tolga
Güllü, Refik
Repetitive Markov processes form a class of processes where the generator matrix has a particular repeating form. Many queueing models fall in this category such as M/M/1 queues, quasi-birth-and-death processes, and processes with M/G/1 or GI/M/1 generator matrices. in this paper, a new iterative scheme is proposed for computing the stationary probabilities of such processes. An infinite state process is approximated by a finite state process by lumping an infinite number of states into a super-state. What we call the feedback rate, the conditional expected rate of flow from the super-state to the remaining states, given the process is in the super-state, is approximated simultaneously with the steady state probabilities. The method is theoretically developed and numerically tested for quasi-birth-and-death processes. It turns out that the new concept of the feedback rate can be effectively used in computing the stationary probabilities.

Citation Formats
T. Tüfekçi and R. Güllü, “An iterative approximation scheme for repetitive Markov processes,” JOURNAL OF APPLIED PROBABILITY, vol. 36, no. 3, pp. 654–667, 1999, Accessed: 00, 2020. [Online]. Available: