Learning of latent class models by splitting and merging components

A problem in learning latent class models (also known as naive Bayes models with a hidden class variable) is that local maximum parameters are often found. The standard solution of having many random starting points for the EM algorithm is often too expensive computationally. We propose to obtain better starting points for EM by splitting and merging components in models with already estimated parameters. This way we extend our previous work, where only a component splitting was used and the need for a component merging was noticed. We discuss theoretical properties of a component merging. We propose an algorithm that learns latent class models by performing component splitting and merging. In the experiments with real-world data sets, our algorithm in a majority of cases performs better than the standard algorithm. A promising extension would be to apply our method for learning cardinalities and parameters of hidden variables in Bayesian networks.