Uncertainty in Selective Bagging: A Dynamic Bi-objective Optimization Model

Bagging is a common approach in ensemble learning that generates a group of classifiers through bootstrapping for classification tasks. Despite its wide applications, generating redundant classifiers remains a central challenge in bagging. In recent years, many selective bagging models have been presented to deal with this challenge. These models mostly focused on the accuracy of classifiers and the diversity among them. Despite the importance of uncertainty in the performance of ensemble classifiers, this criterion has been neglected in selective bagging models. In this paper, we propose a two-stage selective bagging model. In the first stage, we formalize the selective bagging problem as a bi-objective optimization model considering both the uncertainty and accuracy of classifiers. We propose an adaptive evolutionary Two-Arch2 algorithm, named Diverse-Two-Arch2, to solve the bi-objective model. The output of this stage is a subset of classifiers that are diverse, certain about correct predictions, and uncertain about incorrect predictions. While most selective bagging models focus on the selection of a fixed subset of classifiers for all test samples (static approach), our proposed model has a dynamic approach to the selection process. So, in the second stage of the model, we select only certain classifiers to make an ensemble prediction for each test sample. Experimental results on twenty data sets and comparing with two ensemble models, and five state-of-the-art dynamic selective bagging models show the outperformance of the proposed model. We also compare the performance of the proposed Diverse-Two-Arch2 to alternative evolutionary computation methods.