Unsupervised Model-Free Representation Learning

Numerous control and learning problems face the situation where sequences of high-dimensional highly dependent data are available, but no or little feedback is provided to the learner. In such situations it may be useful to find a concise representation of the input signal, that would preserve as much as possible of the relevant information. In this work we are interested in the problems where the relevant information is in the time-series dependence. Thus, the problem can be formalized as follows. Given a series of observations X-0 ,..., X-n coming from a large (high-dimensional) space chi, find a representation function f mapping chi to a finite space Y such that the series f(X-0) ,..., f(X-n) preserve as much information as possible about the original time-series dependence in X-0 ,..., X-n. For stationary time series, the function f can be selected as the one maximizing the time-series information I-infinity(f) = h(0)(f(X)) - h(infinity)(f(X)) where h0(f(X)) is the Shannon entropy of f(X-0) and h(infinity)(f(X)) is the entropy rate of the time series f(X-0) ,..., f(X-n),.... In this paper we study the functional I-infinity(f) from the learning-theoretic point of view. Specifically, we provide some uniform approximation results, and study the behaviour of I-infinity(f) in the problem of optimal control.