Unbiased Time-Average Estimators for Markov Chains

We consider a time-average estimator fk of a functional of a Markov chain. Under a coupling assumption, we show that the expectation of f(k) has a limit mu as the number of time steps goes to infinity. We describe a modification of f(k) that yields an unbiased estimator f<SIC>(k) of mu. It is shown that f<SIC>(k) is square integrable and has finite expected running time. Under certain conditions, f<SIC>(k) can be built without any precomputations and is asymptotically at least as efficient as f(k), up to a multiplicative constant arbitrarily close to one. Our approach also provides an unbiased estimator for the bias of f(k). We study applications to volatility forecasting, queues, and the simulation of high-dimensional Gaussian vectors. Our numerical experiments are consistent with our theoretical findings.