Reduct Based Ensemble of Learning Classifier System for Real-Valued Classification Problems

Rough set theory has proved efficient for many applications, including finding hidden patterns in data, data reduction, evaluating significance of data, and generating sets of decision rules from data. Recently, it has shown to be effective approach for constructing ensemble learning systems as well. Learning classifier systems are genetics-based machine learning techniques that have recently shown a high degree of competence on a variety of data mining problems. Attempts to improve its generalization capabilities in the literature using ensemble learning lack a systematic and robust techniques for partitioning the problem at hand. It is well known that ensemble performance depends on the problem decomposition technique being used. Rough set based learning classifier system ensemble is proposed in this paper. In this approach, rough set attribute reduction is used to generate a set of reducts, and then a diverse subset of these reducts is selected to train an ensemble of base classifiers. The experiments show that classification accuracy of reduct-based ensemble systems outperforms a single learning classifier system model. It has also shown better performance than either an ensemble of classifiers with all attributes being used or a single classifier trained by a single reduct. It has also shown competitive performance to the random subspace ensemble strategy on the set of real data sets used in the experiments.