A new combinatorial algorithm for large Markov chains

We consider large Markov chains which posses specific decomposable structure, the so called Nearly Completely Decomposable Chains (NCD chains). A new theoretical approach for approximate computations of NCD chains has been recently introduced in [11, 12]. The method of forest expansions gives raise to aggregation algorithms, which approximate effectively the characteristics of Markov chain. The algorithms are based on grouping the states of a Markov chain in such a way that the probability of changing the state inside the group is of greater order of magnitude than interactions between groups. In [5] the algorithm approximating stationary distribution (described by transposed system L(T)x = b, where L is derived from transition matrix) was presented; in this paper we illustrate that combinatorial aggregation in the case of non-transposed system (defining e.g. mean hitting time) is not less effective. This novel approach allows us to treat both types of problems in the unified manner. To our knowledge for the first time an aggregation scheme was used to calculate Markov chain characteristics other than stationary distribution.