Entropy Calculation in Particle Filters

This paper presents a differential entropy calculation method to be used for particle mixtures in particle filters. First it is shown that the exact differential entropy of particle mixtures is minus infinity and therefore useless in practice. The disadvantage of using discrete entropy formulation instead of differential entropy is also explained. Unlike the kernel-based methods in the literature, a Bayes rule based approximation is then proposed. The performance of the algorithm is illustrated on a basic Gaussian mixture example.