ENTROPY AND DATA-COMPRESSION SCHEMES

Some new ways of defining the entropy of a process by observing a single typical output sequence as well as a new kind of Shannon-McMillan-Breiman theorem are presented. Here are two sample results: 1) For a stationary ergodic process let R(n)(xi) = inf{k greater-than-or-equal-to n : xi(k+1)xi(k+2)...xi(k+n) = xi1xi2...xi(n), the a.s. lim(n-->infinity) (log R(n)(xi))/n = entropy of the process. 2) In the Lempel-Ziv parsing, a.s. for n sufficiently large most of xi1 ... xi(n) has been parsed into blocks of size roughly, (log n)/h, where h is the entropy of the process.